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ABSTRACT 


Currently,  the  two  most  commonly  used  night  optical  devices  employed  in 
military  operations  are  the  long- wave  infra-red  sensor  and  the  image  intensified  sensor. 
Recent  advances  in  technology  have  permitted  the  fusion  of  the  output  of  these  two 
devices  into  a  single  color  display  that  potentially  combines  the  capabilities  of  both 
sensors  while  overcoming  their  limitations.  Although  the  concept  is  appealing,  previous 
sensor  fusion  studies  have  been  inconclusive  on  the  benefits  of  an  artificially  colored 
target.  Perhaps,  an  artificially  colored  target  disrupts  an  operator’s  visual  processing 
thereby  hindering  the  detection  of  a  target.  The  purpose  of  this  thesis  is  to  compare  the 
effects  of  artificial  color,  natural  color,  and  monochrome  formats  in  visual  scene 
perception.  It  is  hypothesized  that  participant  response  times  and  error  rates  would  be 
greater  at  detecting  an  artificially  colored  target  compared  to  a  natural  colored  or  a  target 
presented  achromatically.  Two  experiments  were  conducted.  Experiment  1  used  non- 
degraded  imagery  and  Experiment  2  used  degraded  imagery  to  compare  these  effects.  It 
was  found  that  reaction  time  and  error  rates  for  naturally  colored  and  achromatic  images 
were  similar  and  substantially  less  when  compared  to  artificially  colored  images.  For 
degraded  scenes,  natural  color  was  more  beneficial  when  compared  to  achromatic  and 
artificially  colored  scenes.  Additionally,  artificially  colored  scenes  caused  extremely 
large  error  rates  and  reaction  times.  These  results  will  provide  algorithm  developers 
insight  into  the  importance  of  color  constancy. 
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EXECUTIVE  SUMMARY 


Recently,  military  requirements  have  determined  the  need  to  improve  existing 
night  vision  devices  used  in  military  applications.  Shortfalls  currently  exhibited  by  these 
operational  night  vision  devices  are  the  lack  of  target  and  environment  contrast,  poor 
display  resolution,  inadequate  magnification  properties  and  small  fields  of  view. 
Technology  now  exists  that  can  provide  a  modification  to  existing  night  vision  systems 
by  replacing  them  with  a  dual-band  color  fused  sensor.  Modifying  these  systems  is  more 
cost  effective  than  the  complete  replacement.  Incorporating  the  features  offered  by  the 
dual-band  color  fused  sensor  into  current  military  hardware  may  significantly  improve 
the  aforementioned  limitations.  Extensive  research  in  visual  search  experiments  has 
demonstrated  the  utility  of  sensor  fusion;  however,  the  advantages  are  not  clearly  defined. 
It  has  been  determined  that  color  fusion  may  benefit  target  detection  but  hinders  a 
participant’s  situational  awareness.  Evaluation  provided  by  various  operators  have 
indicated  that  the  lack  of  color  constancy  of  the  fused  devices  caused  the  scenes  to  be 
aesthetically  displeasing. 

Vision  scientists  have  long  hypothesized  that  color  enhances  visual  performance 
in  object  recognition.  Studies  have  shown  that  color  plays  a  role  in  object  recognition, 
but  the  significance  of  color  has  often  been  debated.  Color  is  commonly  used  to  segment 
images  into  regions  that  contain  information  about  specific  objects  within  the  scene. 
However,  it  has  generally  been  demonstrated  that  object  recognition  is  based  primarily  on 
the  representation  of  shape  and  that  identification  by  surface  texture  and  color  may 
therefore  be  used  as  secondary  cues  in  identification.  With  this  theory  in  mind,  we  must 
ask  why  the  military  would  want  to  incorporate  color  into  existing  night  vision  devices. 
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Space  requirements  in  military  vehicles  and  aircraft  have  limited  the  size  of  the 
visual  displays.  These  small  displays  present  imagery  that  is  often  blurred  or  viewed 
under  low  light  conditions.  Images  that  suffer  from  this  form  of  degradation  lose  their 
visual  acuity  and  edge  definition.  Research  concerning  colors  role  using  degraded 
objects  has  demonstrated  that  color  is  important  in  object  identification.  Without  defined 
object  contour,  color  is  used  to  segment  objects  from  their  background  to  facilitate 
identification. 

One  significant  limitation  attributed  to  night  vision  fused  sensors  is  that  current 
algorithms  are  incapable  of  providing  consistent  color  to  objects.  This  is  because 
information  taken  from  night  sensors  is  normally  outside  the  range  of  the  HVS's 
sensitivity.  Therefore,  color  is  added  according  to  the  existing  contrast  between  the 
object  and  background  temperature  differential.  The  temperature  differential  dictates  the 
coloring  scheme  of  the  object,  thereby  prohibiting  any  form  of  standardization  amongst 
identifying  objects  by  specific  colors.  However,  false  coloring  algorithms  are  currently 
widely  used  in  medical  and  meteorological  displays.  These  diverse  areas  have  devised 
color  schemes  as  symbolic  measures  of  significance.  With  the  inability  to  standardize 
object  color  in  fused  night  vision  sensors,  will  color  enhance  object  recognition? 

The  influence  of  color  was  studied  as  a  feature  against  its  influence  as  stored 
knowledge  in  object  recognition.  Stored  knowledge  is  interpreted  as  semantic 
information  that  explains  the  prototypical  colors  of  objects,  such  as  the  knowledge  that 
apples  are  typically  red  or  green.  Participants  respond  faster  to  objects  having 
prototypical  colors  than  objects  that  do  not  possess  any  correlation  to  the  presented  color. 
When  objects  were  viewed  in  an  unnatural  scene,  participant  reaction  times  increased. 
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This  thesis  is  designed  to  validate  previous  color  studies,  by  using  professional 
photographs  with  and  without  degraded  edges.  It  is  hypothesized  that  in  high  resolution 
scenes,  natural  color  images  will  not  enhance  object  recognition  when  compared  to 
monochromatic  images,  and  non-prototypical  (artificial)  colored  scenes  will  hinder  object 
recognition.  For  low-resolution  images,  natural  color  will  enhance  object  recognition 
when  compared  to  monochrome  scenes,  and  artificially  color  scenes  would  hinder  object 
recognition. 

The  results  of  both  experiments  confirm  the  hypothesis.  Participant’s  that  viewed 
high  resolution  natural  colored  and  monochromatic  scenes  had  similar  reaction  times  and 
error  rates.  In  the  artificially  colored  scenes,  the  reaction  times  and  error  rates  were 
substantially  larger.  For  low-resolution  scenes,  natural  color  significantly  enhanced 
object  recognition  while  artificial  color  hindered  a  participant's  ability  to  efficiently 
recognize  the  objects.  Again,  participant  error  rates  paralleled  their  reaction  times. 
Participants  were  more  efficient  when  viewing  natural  color  scenes  and  were 
significantly  inefficient  when  viewing  scenes  that  did  not  possess  any  form  of  consistent 
color. 

This  research  has  confirmed  previous  vision  studies  that  included  the  use  of 
objects  being  observed  in  their  natural  environments  instead  of  against  a  neutral 
background.  It  is  recommended  that  similar  vision  studies  be  conducted  using  imagery 
taken  from  color  fused  sensors  instead  of  commercial  imagery  to  further  validate  these 
findings. 
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I. 


INTRODUCTION 


Throughout  the  history  of  warfare,  nations  have  pursued  methods  that  helped 
them  gain  tactical  advantage  over  the  enemy.  Many  of  these  methods  involved  the 
element  of  surprise.  However,  because  of  advancements  in  satellite  surveillance  systems 
and  other  forms  of  innovations  in  intelligence  gathering,  the  element  of  surprise  has  been 
effectively  reduced.  A  remaining  element  of  warfare  that  has  yet  to  be  fully  exploited  is 
the  ability  to  operate  under  the  cover  of  darkness,  or  in  low  visibility.  Most  intelligence 
and  satellite  surveillance  systems  parallel  the  human  visual  system  (HVS)  in  that,  they 
educe  reflected  light,  and  as  such,  are  ineffective  at  night.  Therefore,  military  technology 
has  enunciated  the  development  and  refinement  of  night  vision  systems  that  permit  forces 
to  operate  under  the  cover  of  darkness.  These  systems  are  commonly  referred  as  Night 
Vision  Devices  (NVD's). 

The  two  most  common  NVD's  employed  by  military  forces  are  the  forward- 
looking  infra-red  radar  (FLIR)  and  the  night  vision  goggle  (NVG).  Although  their 
purpose  is  to  expose  environmental  information  in  the  darkness,  they  function  very 
differently.  FLIR's  operate  in  the  infra-red  region  of  the  electromagnetic  spectrum.  They 
detect  thermal  differences  between  an  object  and  its  background.  The  greater  the  thermal 
difference,  the  higher  the  contrast  portrayed  in  the  image.  Hence,  a  major  limitation  of  a 
FLIR  sensor  is  that  it  requires  a  thermal  difference  to  adequately  reveal  a  target  that  is 
detectable  to  a  human  observer.  There  are  numerous  occasions  where  this  limitation  may 
impede  night  operations.  Heavy  precipitation  and  dense  foliage  tend  to  create  a 
homogeneous  thermal  scene.  When  this  occurs,  FLIR  sensors  are  superfluous  to  night 
operators  (MAWTS-1, 1995). 
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NVG's  require  reflected  radiated  light  to  function.  Whether  this  energy  originates 
from  celestial  radiated  energy  or  man-made  energy,  NVG's  magnify  this  energy  to 
illuminate  the  scene.  A  major  limitation  of  this  device  is  the  requirement  of  reflected 
energy.  Operations  under  cloud  cover  or  in  dense  foliage  may  severely  restrict  the 
devices  potential  (Wolfe,  1994). 

U.S.  military  forces  have  successfully  employed  these  devices  in  combat 
operations.  As  recently  as  in  Operation  Desert  Storm  and  Bosnia  peacekeeping 
operations,  military  forces  have  used  NVD's  frequently  and  have  conducted  extensive 
night  operations  safely  and  effectively  (Time-Life  Books,  1991).  Unfortunately,  the 
widespread  proliferation  of  this  technology  (Klass,  1994)  has  permitted  other  nations  to 
acquire  these  devices  which  has  reduced  the  technological  superiority  of  U.S.  forces  on 
the  battlefield.  In  order  to  regain  battlefield  superiority,  the  United  States  is  pursuing 
technology  that  may  enable  forces  to  reacquire  this  advantage.  Some  of  these  advantages 
may  come  from  improving  the  existing  night  vision  capability. 

Computer  technology  has  permitted  the  U.S.  to  consolidate  sensory  outputs  from 

a  combination  of  different  NVD’s  within  a  single  representation.  One  such  combination 

consists  of  fusing  the  output  of  the  NVG  sensor  with  the  output  of  the  FLIR  sensor. 

* 

Individually,  each  sensor  displays  unique  characteristics  of  the  visual  scene.  Despite  the 
deficiencies  of  each  sensor,  fusing  individual  sensor  outputs  into  a  single  representation 
may  potentially  overcome  individual  weaknesses  and  provide  more  opportunities  to 
observe  the  scene  under  more  heterogeneous  environmental  conditions.  For  example,  in 
the  absence  of  visible  light,  an  object  may  still  be  observed  if  it  possess  any  form  of 
thermal  signature.  Although  sensor  fusion  may  potentially  take  advantage  of  individual 
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sensor  limitations,  the  output  of  each  sensor  is  presented  in  the  same  monochromatic 
format  that  is  displayed  with  individual  sensors.  Because  of  this,  a  lack  of  background 
contrast  may  continue  to  exist.  One  possible  method  of  subjugating  this  lack  of 
background  contrast  is  by  introducing  color  to  the  display.  Color  may  intensify  the 
context  of  a  scene,  thereby  permitting  the  operator  more  opportunity  to  efficiently 
perform  his  search  (Krebs,  Scribner,  Miller,  Ogawa,  and  Shuler,  1998). 

These  color  fusion  algorithms  presume  that  the  addition  of  color  to  the  scene 
would  enhance  object  detection  (Scribner,  Warren,  Schuler,  Satyshur  and  Kruer,  1998). 
One  significant  limitation  to  providing  color  to  fused  images  is  the  inability  to  reconstruct 
color  constancy.  Current  computer  algorithms  provide  image  colors  according  to  thermal 
contrast.  Because  the  thermal  signature  is  continuously  changing,  the  inability  to  control 
for  color  constancy  is  lost.  Recent  studies  comparing  the  possible  benefits  of  employing 
color  in  displays  with  monochromatic  displays  have  been  inconsistent.  These  studies 
either  failed  to  show  a  significant  advantage  for  color  over  grayscale  images  or  they 
found  the  advantage  to  be  scene  dependent.  Therefore,  the  research  failed  to  find  specific 
advantages  to  displaying  fused  color  scenes.  (Sampson,  Krebs,  Scribner,  and  Essock, 
1996;  Krebs,  et  al.,  1998;  Essock,  Sinai,  McCarley,  Krebs,  and  DeFord,  1998) 

Similar  false  coloring  algorithms  are  currently  being  used  in  medical  (Schmidt, 
Hier,  Benyamin,  and  DeForest,  1988),  meteorological  (Uttal,  Baruch  and  Allen,  1994), 
and  automotive  displays  (Krebs,  Scribner,  Schuler,  Miller  and  Lobek,  1996).  These 
diverse  areas  have  devised  color  scenes  as  symbolic  measures  of  significance.  For 
example,  the  color  red  displayed  on  a  weather  radar  screen  portrays  the  existence  of 
severe  precipitation  whereas  the  green  may  signify  light  precipitation.  Research  and 
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training  in  these  fields  have  standardized  these  false  colors  into  providing  meaningful 
information  to  individuals.  It  is  possible  that  if  a  similar  standardization  of  color  is 
incorporated  in  image  fusion  algorithms,  the  meaning  of  specific  colors  could  be  learned 
and  therefore  proven  beneficial. 

NVD’s  are  single-band  sensors  that  operate  outside  the  range  of  the  human  visual 
system's  sensitivity.  They  convey  no  chromatic  information.  Current  artificial  color 
algorithms  base  their  coloring  schemes  on  sensor  contrast  between  the  reflective  and 
thermal  energy  detected  from  the  scene  (Krebs,  et  al.,  1998).  The  objective  of  artificial 
color  processing  is  to  emphasize  sensor  contrast  rather  than  reproducing  the  actual  or  true 
colors  of  the  scene.  As  a  result,  the  “correct”  color  information  of  the  scene  is  lost. 
Furthermore,  the  sensor  contrast  differential  dictates  the  coloring  scheme  of  the  image, 
thereby  prohibiting  the  ability  to  provide  standard  colors  to  objects  within  a  scene. 
Perhaps  it  is  this  lack  of  color  standardization  that  is  a  contributing  factor  to  the 
inconclusive  results  of  previous  psychophysical  fused  color  studies  (Steele  and  Perconti, 
1997). 

Basic  research  on  object  recognition  has  determined  color  to  be  non-beneficial 
when  compared  to  grayscale  images  (Biederman  and  Ju,  1988).  However,  this  inference 
has  often  been  debated.  Some  research  has  shown  that  color  benefits  object  recognition 
while  others  have  not  shown  any  significance.  A  possible  reason  for  these 
inconsistencies  may  arise  from  the  differing  experimental  methodologies  employed  by 
these  studies.  It  may  be  the  type  of  task  performed  and  the  cues  that  were  provided  to 
participants  that  determined  the  role  of  color  (Wurm,  L.H.,  Legge,  G.E.,  Isenberg,  L.M., 
and  Luebker,  A.,  1993;  Biederman  and  Ju,  1988;  Joseph  and  Proffitt,  1996).  No  known 
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research  has  attempted  to  determine  colors  role  using  global  scenes  in  object  recognition. 
Therefore,  attempting  to  apply  the  existing  color  theories  to  pseudo-color  fused  theory 
may  be  inappropriate. 

A.  OBJECTIVE 

The  objective  of  this  thesis  is  to  clarify  the  role  of  natural  color  and  artificial  color 
in  natural  scene  perception.  Currently,  visual  devices  in  military  vehicles  are  normally 
small,  compact  and  are  often  characterized  by  poor  resolution.  This  may  cause  imagery 
to  be  presented  as  blurred  and  lack  defined  contour  information.  Under  these  additions,  it 
might  be  expected  that  color  would  benefit  object  recognition.  This  leads  us  to  question 
the  function  of  color  vision,  and  the  role  it  plays  in  object  recognition,  especially  if  the 
displayed  color  does  not  depict  the  natural  scene. 

B.  THE  RESEARCH  QUESTION 

A  major  reason  that  previous  research  concerning  the  role  of  color  in  object 
recognition  has  been  inconclusive  may  be  due  to  the  existing  experimental 
methodologies.  The  methodologies  of  previous  research  ranged  from  comparing  black 
and  white  line  drawings  to  colored  pictures  of  manufactured  objects  (Biederman  and  Ju, 
1988),  to  comparing  natural  and  artificially  colored  line  drawings  of  natural  objects 
against  neutral  backgrounds  (Joseph  and  Proffitt,  1996).  It  may  be  concluded  that 
different  methodologies  may  produce  varied  results.  In  this  thesis,  objects  will  be 
observed  in  their  natural  settings  with  the  intent  to  compare  the  effects  of  natural  and 
artificial  color  images  using  professionally  photographed  images.  Once  participant 
performance  is  known  when  viewing  non-degraded  color  images,  it  is  expected  that  the 
results  will  adversely  differ  when  viewing  degraded  images. 
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C.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

For  this  research,  professionally  photographed  daylight  images  were  used  in  place 

of  fused  images  to  determine  the  role  of  true  and  artificial  color  in  object  recognition. 
This  provided  more  flexibility  in  selecting  images  based  upon  standards  that  were  set 
prior  to  data  collection.  Although  the  selected  images  did  not  resemble  fused  imagery 
from  NVD’s,  the  role  of  true  color  and  artificially  colored  natural  scenes  could  still  be 
examined.  Two  experiments  were  conducted.  The  first  experiment  attempted  to 
determine  whether  artificial  color  was  perceived  similarly  as  monochrome  and  natural 
color  using  non-degraded  imagery.  Based  on  previous  findings,  it  is  hypothesized  that 
there  will  be  no  advantage  of  color  in  object  recognition,  but  artificially  colored  scenes 
may  disrupt  the  HVS's  ability  to  efficiently  perceive  objects  within  the  scene.  The 
second  experiment  attempted  to  simulate  NVD  representations  using  similar  commercial 
images  used  in  the  first  experiment.  However,  noise  was  added  to  the  imagery  to 
approximate  the  degraded  resolution  of  NVD  displays.  To  validate  the  research 
conducted  by  Biederman  and  Ju  (1988),  Joseph  and  Proffitt  (1996)  and  Wurm,  et  al., 
(1993),  it  is  hypothesized  that  there  will  be  an  advantage  of  color  in  object  recognition 
and  that  artificially  colored  images  will  disrupt  the  HVS  when  observed  in  degraded 
conditions.  Additionally,  given  the  additional  data  obtained  in  the  second  experiment 
using  different  imagery,  portions  of  the  findings  from  the  first  experiment  could  be 
validated. 

D.  ORGANIZATION  OF  STUDY 

This  thesis  is  organized  into  five  chapters.  Chapter  I  is  the  introduction,  which 
gives  a  general  description  of  current  limitations  to  providing  color  to  fused  images. 
Chapter  II  provides  the  necessary  background  information  relevant  to  night  vision 
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technology.  It  also  provides  in  detail,  previous  research  of  the  role  of  color  in  object 
recognition.  Chapter  IE  and  IV  describe  the  methodology  used  to  conduct  Experiments  1 
and  2  respectively,  the  procedures  used  in  selecting  and  manipulating  the  images,  and  the 
experimental  design.  The  chapters  also  provide  data  analysis  and  interpretation  of  the 
data  collected  from  both  experiments.  Chapter  V  provides  conclusions  and 
recommendations  for  further  research. 
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II.  BACKGROUND 


A  general  historical  review  of  events  leading  up  to  the  technology  that  produced 
these  devices  should  provide  the  reader  with  a  basic  understanding  of  the  problems 
associated  with  providing  color  to  NVD’s.  A  general  description  and  characteristics  of 
individual  sensors  is  provided.  With  knowledge  of  sensor  capabilities  and  limitations,  the 
characteristics  of  sensor  fusion  can  be  understood.  Also,  a  comprehensive  understanding 
of  previous  research  involving  the  use  of  color  in  object  recognition  is  provided  to  aid  in 
the  development  of  theories  that  may  help  provide  answers  to  the  contradictions  exhibited 
in  previous  color  fusion  studies. 

A.  HISTORY 

To  be  successful  in  warfare,  a  striking  force  must  maintain  an  advantage  over  the 
adversary  (Clausewicz,  1976).  Efforts  to  maintain  this  advantage  have  resulted  in  the 
development  of  more  destructive  weaponry  and  the  creation  of  more  efficient  tactics. 
Although  many  new  technologies  were  introduced  during  World  War  II,  American  forces 
did  not  exploit  night  combat  operations  until  the  latter  stages  of  the  war.  Until  then,  night 
operations  were  typically  utilized  for  planning  subsequent  daytime  operations  and 
relocating  forces  for  an  ensuing  battle.  Many  U.S.  forces  were  lost  in  combat  during 
daylight  hours.  It  was  not  until  the  “Battle  at  Cape  St.  George”  during  the  Pacific 
campaign  in  World  War  n,  where  Captain  (later  Admiral)  Arleigh  Burke’s  forces 
inflicted  heavy  damage  upon  the  main  Japanese  surface  forces,  that  large-scale  night 
tactics  were  designed  and  practiced.  One  of  the  major  contributing  factors  to  Burke’s 
success  was  his  ability  to  exploit  night  operations.  He  recognized  this  paradigm  and 
prepared  his  forces  by  devising  and  employing  tactics  that  took  advantage  of  the  enemy’s 
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inability  to  respond  under  these  conditions  (Potter,  1990). 

The  Korean  War  produced  no  radical  changes  to  combat  tactics  during  night 
operations.  This  was  partially  due  to  the  insufficiency  of  technical  advances  of  night 
optical  devices,  and  the  fairly  limited  understanding  that  researchers  had  of  human  visual 
processing.  However,  since  then,  the  infra-red  industry  has  expanded  rapidly.  This 
industrial  development  was  motivated  by  the  inability  of  U.S.  forces  to  expostulate  North 
Vietnamese  forces  from  conducting  night  operations  against  isolated  outposts,  force 
resupply,  or  moving  logistics  (Schwarzkopf,  1992).  Since  the  post  Vietnam  era,  virtually 
all  military  high  value  platforms  possess  infra-red  imaging  capability.  The  vast 
investment  in  NVD’s  by  the  military  services  and  law  enforcement  agencies  has  spawned 
an  enormous  industrial  community  dedicated  not  only  to  the  manufacture  of  the  devices, 
but  also  to  their  large-scale  integration  into  weapons,  ships  and  combat  vehicles.  As  a 
result,  researchers  who  analyze,  measure  and  predict  the  performance  of  NVD’s  have 
grown  from  a  small  community,  intimately  involved  and  thoroughly  familiar  with  the 
technology  and  its  theory,  to  an  extremely  diverse  population  of  engineers,  managers  and 
scientists.  The  necessity  to  see  “into  the  night”  is  the  driving  force  in  the  development  of 
NVD’s  (Berry,  1993). 

There  are  numerous  night  vision  enhancement  systems  in  the  U.S.  armed  forces’ 
inventory.  Every  system  is  capable  of  functioning  beyond  the  HVS's  unaided  capability. 
These  systems  have  been  proven  effective  in  a  myriad  of  diverse  combat  operations. 
However,  unanticipated  problems  have  arisen  while  utilizing  these  devises.  A  human’s 
unaided  perception  of  the  surroundings  at  night  is  vastly  different  when  observed  with 
NVD’s.  The  user’s  lack  of  understanding  of  the  night  environment  and  its  impact  on  the 
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NVD’s  performance  has  caused  the  capabilities  of  these  devices  to  be  exceeded,  resulting 
in  numerous  mishaps  (Salvendy,  1997). 

Currently,  almost  all  military  personnel  have  trained  with  or  have  been  exposed  to 
NVD’s.  With  the  radical  change  in  military  budgets  and  huge  reduction  of  forces,  the 
exploitation  of  night  operations  allows  U.S.  military  forces  to  do  “more  with  less”  while 
retaining  the  combat  advantage  over  enemy  forces.  As  recently  seen  in  Operation  Desert 
Storm,  many  of  U.S.  military  campaigns  occurred  at  night.  Coalition  forces  exposed 
Iraqi  forces  to  a  “24  hour  battlefield”  by  employing  NVD  tactics;  a  strategy  the  Iraqi 
forces  were  ill  prepared  to  confront.  U.S.  forces  easily  demonstrated  the  ability  to 
conduct  sophisticated  and  coordinated  operations  with  the  aid  of  these  devices  on  their 
platforms  (Schwarzkopf,  1992).  A  brief  overview  of  the  NVD’s  is  presented  to  provide 
the  reader  with  a  basic  understanding  of  these  devices.  A  more  detailed  coverage  of  the 
theory  and  operation  of  NVD’s  is  provided  in  MAWTS-1  (1994). 

B.  IMAGE  INTENSIFIERS  (NVG’S) 

NVG’s  operate  in  the  near  visible  and  near  infra-red  (IR)  spectrum  (wavelengths 

of  570-900  nanometers)  and  depend  entirely  on  reflected  energy  (Figure  1). 
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Figure  1:  The  portions  of  the  electromagnetic  spectrum  used  in  unaided  human 


vision,  NVG’s  and  FLIR's.  (MAWTS-1, 1995) 

The  first  generation  NVG's  relied  entirely  on  light  at  visible  wavelengths,  but  the 
third  generation  (GEN-DI)  instruments  make  use  of  infra-red  light  just  beyond  the  visible 
spectrum  (NVESD,  1997).  Current  NVG  technology  centers  on  the  GEN-III  devices. 
Even  on  the  darkest  of  nights  when  visible  light  is  practically  non-existent,  infra-red 
energy  is  still  plentiful.  Additionally,  the  reflective  characteristics  of  different  surfaces 
are  more  pronounced  in  the  infra-red  band,  thus  creating  an  image  displayed  with  greater 
contrast  (Time-Life  Books,  1991). 

The  two  sources  of  light  that  enhance  the  HVS  aided  by  NVG’s  are  reflected  and 
generated  light.  Reflected  light  normally  originates  from  energy  emitted  by  celestial 
bodies  such  as  the  moon  or  stars  that  is  reflected  off  an  object's  surface.  Generated  light 
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originates  from  artificial  sources  of  illumination  such  as  city  lights  and  shipboard  lights. 
A  specific  characteristic  of  NVG's  is  that  with  proper  ambient  light  levels,  the  visual 
clarity  is  superb.  There  are  several  variables  that  affect  the  optimal  performance  of 
NVG’s.  These  variables  include  the  amount  of  illumination  within  the  scene, 
characteristics  of  the  terrain,  and  atmospheric  conditions  in  which  the  sensor  is  operating 
(FM  34-81-1,  1992).  Atmospheric  conditions  have  the  greatest  impact  on  NVG 
effectiveness.  The  dominant  atmospheric  factor  that  affects  NVG  performance  is  the 
quantity  of  water  vapor  (humidity)  present.  This  is  because  water  molecules  in  the  air 
absorb  and  reflect  light  from  the  intended  source  (FM  34-81-1,  1992).  A  sample  image 
viewed  with  a  NVG  is  shown  in  Figure  2. 


Figure  2:  Sample  image  taken  with  a  NVG.  Notice  the  clarity  of  the  objects. 

(A.P.Hill,  1998) 


C.  THERMAL  SENSORS  (FLIR) 

FT  .re’s  are  electronic  devices  that  convert  invisible  thermal  energy  into  a  visible 
image.  FT. re’s  operate  in  the  7-14  micron  wavelength  band  of  the  electromagnetic 
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spectrum  (Figure  1).  FLIR  technology  is  based  on  the  detection  of  emitted  and  reflected 
thermal  energy.  FLIR’s  operate  as  differential  temperature  measuring  systems,  in  that,  a 
measurable  temperature  difference  must  exist  between  an  object  and  the  background  in 
which  the  sensor  operates.  This  measurement  is  referred  to  as  “DELTA  T” 
(Schlessinger,  1995). 

The  infra-red  signatures  of  targets  and  backgrounds  are  far  more  complex  than  the 
visible  signatures  in  NVG’s  because  emission  has  virtually  no  influence  (NVESD,  1997). 
As  complex  as  infrared  signatures  are,  the  limited  scientific  capabilities  at  the  disposal  of 
the  FLIR  community  cannot  cope  with  anything  beyond  the  simplest  expression  of  target 
and  background  signature.  The  predominant  analysis  in  thermal  theory  is  currently  based 
on  the  assumption  that  an  object  behaves  like  a  blackbody  at  a  temperature  some  number 
of  degrees  different  from  a  uniform  blackbody  background,  such  as  the  environment  in 
which  the  object  is  viewed  (Campana,  1993). 

The  two  general  types  of  sources  that  enable  FLIR  devices  to  function  are  thermal 
and  selective  radiated  thermal  energy.  Thermal  sources,  such  as  the  sun  or  combustible 
engine,  radiate  over  a  wide  frequency  spectrum  with  the  maximum  radiated  energy  level 
concentrated  at  a  particular  frequency.  Selective  thermal  sources  generally  concentrate 
on  a  very  narrow  frequency  band.  An  example  of  selective  thermal  radiation  is  a  laser 
beam  (Steele  and  Perconti,  1997). 

The  diurnal  cycle  is  a  measure  of  the  rate  at  which  objects  and  their  backgrounds 
heat  and  cool.  Large  dense  objects  tend  to  heat  and  cool  slowly,  whereas,  less  dense 
objects  such  as  trees  and  grass,  heat  and  cool  more  rapidly.  Objects  heated  by  sunlight 
are  the  mechanism  through  which  the  diurnal  cycle  drives  thermal  signatures  (Shumaker, 
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Wood,  and  Thacker,  1988).  The  rate  of  change  in  temperature  of  a  surface  is  dependent 
not  only  on  its  emissivity  but  also  on  its  shape,  size,  heating  capacity,  conductivity  and 
convective  cooling/heating  effects  (Wolfe,  1996). 

Sometimes,  the  radiated  temperature  of  an  object  matches  the  temperature  of  the 
surrounding  environment  resulting  in  a  Delta  T  =  0.  On  hot  days,  an  object  normally 
fails  to  reach  the  evening  background  temperature  before  the  sun  rises  again  to  restore  the 
heat  cycle.  As  the  sun  rises,  the  cooler  background  warms  up  rapidly  and  soon  exceeds 
the  target  temperature.  Twice  in  a  24-hour  period.  Delta  T  goes  below  zero.  From 
sunrise  to  late  afternoon,  the  background  temperature  is  generally  greater  than  the  object 
temperature  thereby  masking  the  objects  thermal  effect  on  a  FLIR  device  (FM  34-81-1, 


Figure  3:  The  Diurnal  cycle  for  a  man-made  object  and  background  terrain. 
Crossover  times  are  also  shown  (MAWTS-1, 1994) 

Atmospheric  conditions  also  have  a  very  profound  effect  on  the  effectiveness  of 
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FLIR’s.  The  two  predominate  atmospheric  factors  are  energy  absorption  and  scattering. 
Thermal  absorption  in  the  atmosphere  is  a  greater  concern  that  contributes  to  thermal 
signature  loss  and  scatter  (MAWTS-1,  1994).  A  characteristic  of  FLIR  representation  is 
the  amount  of  contrast  level  between  objects  and  backgrounds.  This  contrast  dictates  the 
resolution  of  the  display.  A  sample  image  viewed  with  a  FLIR  is  shown  in  Figure  4. 


Figure  4:  Sample  image  taken  with  a  FLIR.  Notice  the  brightness  and  poorer 
resolution  of  the  objects.  (A.P.HU1, 1998) 


Current  FLIR  technology  is  centered  on  first  generation  (GEN-I)  FLIR  devices. 
The  U.S.  Army  began  integration  of  second  generation  FLIR’s  into  new  and  existing 
weapon  systems  to  maximize  U.S.  forces  advantage  on  the  battlefield.  The  center  piece 
of  this  system  is  a  common  subsystem  that  is  currently  being  produced  for  the  Abrams 
Tank,  Bradley  Fighting  Vehicle,  and  Long  Range  Advanced  Scout  Sensor  System 
(NVESD,  1997). 

D.  FUSED  SENSORS 

Improvements  to  individual  NVD’s  have  provided  substantial  increases  in  sensor 
detection  ranges  and  display  resolution.  However,  these  improvements  cannot  avoid  the 
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limitations  of  the  electromagnetic  spectrum.  Each  sensor  suffers  from  its  own 
limitations.  NVG’s  must  have  reflected  light  for  detection  and  FLIR's  must  be  able  to 
detect  a  contrast  in  thermal  energy  between  an  object  and  the  background.  Scene 
analysis  by  an  operator  may  potentially  benefit  from  a  fused  representation  (combined 
FLIR  and  NVG)  of  a  scene  taken  in  different  spectral  bands.  For  example,  following  a 
period  of  extensive  cooling  such  as  a  rain  shower,  the  visible  bands  may  represent  the 
background  in  great  detail  whereas  the  infra-red  bands  that  represent  the  scene  may  be 
negligible  or  provide  a  representation  of  the  scene  that  is  severely  degraded  due  to  the 
low  thermal  contrast.  Also,  a  target  that  is  camouflaged  for  visible  detection  may  not  be 
detected  in  the  visible  bands,  but  may  be  clearly  represented  in  the  infra-red  bands  by 
providing  a  thermal  contrast  between  the  object  and  background.  The  fusion  of  visible 
and  thermal  imagery  on  a  single  display  may  allow  both  target  detection,  provided  by  the 
thermal  image  with  respect  to  the  context,  and  unambiguous  target  localization,  provided 
by  the  visible  image  (Toet,  Uspeert,  Waxman,  and  Aguilar,  1997).  Technology  now 
exists  that  enables  information  provided  by  these  sensors  to  be  combined  and  represented 
on  a  single  display. 

The  basis  for  fusing  multiple  sensors  originated  from  the  notion  that  pit  vipers 
incorporate  a  similar  concept.  Various  species  of  pit  vipers  combine  the  attributes  of 
visible  and  infra-red  vision  for  hunting  at  night  with  little  or  no  visible  light.  The  snake’s 
visual  system  is  composed  of  infra-red  sensors  and  pit  organs,  located  near  the  head,  that 
open  below  and  in  front  of  the  eyes.  Infra-red  information  is  sensed  by  the  snake’s  pit 
organs  and  is  then  sent  to  the  brain,  where  it  is  combined  with  visible  information 
obtained  by  the  snakes  eyes  (Hartline  and  Newman,  1982).  This  combined  information 
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allows  the  pit  viper  to  hunt  efficiently  in  daytime  as  well  as  at  night.  Although  the  pit 
viper  can  strike  with  deadly  accuracy  on  just  the  thermal  signature  of  the  prey,  mapping 
of  neurological  impulses  in  the  brain  show  that  the  visual  and  infra-red  spectrum  are 
combined  (fused)  to  provide  a  complete  picture  of  the  scene  (Hartline  and  Newman, 
1982). 

Having  the  capability  to  combine  multiple  NVD’s  contained  in  military 
equipment,  ongoing  research  is  being  conducted  to  attempt  to  develop  an  optimum  multi¬ 
sensor  integration  suite  (Campana,  1993).  Some  advantages  of  combining  multiple 
spectrums  into  a  single  representation  might  be: 

•  Reduced  cost,  space,  and  weight  requirements  from  combined  resources. 

•  Reduced  operator  workload  by  eliminating  the  need  to  alternate  between  two 
sensors. 

•  Improved  object  search,  detection,  and  recognition. 

One  of  the  characteristics  of  the  HVS  is  its  ability  to  process  color  efficiently 
(Jayle,  Ourgurd,  Baisinger,  Holmes  and  Duke-Elder,  1959).  Presenting  displays  in 
monochromatic  format,  such  as  the  displays  of  NVD’s,  restricts  the  capability  to 
optimally  utilize  the  HVS.  If  the  HVS  were  monochromatic,  scenes  would  be  similar  to 
those  displayed  on  a  cathode-ray  tube  (CRT),  where  digitized  monochromatic  images 
physically  represent  each  pixel  as  one  of  256  different  levels  of  brightness  (gray-levels) 
(Waxman,  Fay,  Gove,  Seibert,  Racamato,  Carrick,  and  Savoye,  1995).  However,  the 
HVS  can  perceptually  distinguish  only  about  100  different  gray  levels  on  a  CRT  with 
images  collected  under  ideal  lighting  conditions  (Scribner,  et  al.,  1998).  For  images 
collected  under  dim  light,  the  gray-scale  depth  of  the  image  is  even  more  limited  (Toet 
and  Walraven,  1996).  To  improve  the  probability  of  detecting  an  object  within  a  scene. 
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combining  multiple  bands  into  a  single  fused  color  scene  should  enhance  the  contrast 
between  the  foreground  and  background  objects  in  the  scene  (Scribner,  Satyshur  and 
Kruer,  1993).  Figure  5  shows  an  example  of  an  image  viewed  under  fused  color. 


Figure  5:  Sample  image  taken  with  fused  sensors  (A.P.HH1,  1998). 


Besides  the  advantages  already  discussed  of  displaying  fused  sensors  on  a  single 
representation,  fusion  also  provides  the  opportunity  for  the  presentation  of  imagery  on 
chromatic  displays.  However,  color  imaging  is  only  possible  when  the  responses  of  the 
fused  sensors  differ  in  spectral  sensitivity  (Krebs,  et  al.,  1998). 

It  is  important  to  understand  the  distinction  between  a  color  scene  observed  by  the 
unaided  eye  through  the  human  observer  and  a  processed  pseudo-color  scene. 
Psychologically,  certain  colors  have  developed  meaning  with  time.  Red  for  instance, 
possesses  an  instinctive  psychological  impact  that  tends  to  draw  attention,  enhances 
detection  and  discriminates  possible  dangers  (Valdez  and  Mehrabian,  1994).  In  order  to 
ensure  that  these  color  cues  appear  the  same  for  a  wide  range  of  environments  throughout 
the  day,  the  visual  system  adapted  the  ability  to  distinguish  these  colors  under  varying 
natural  lighting  conditions  (Sekuler  and  Blake,  1994). 
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The  spectral  composition  of  light  impinging  on  the  HVS  sensory  receptors  is 
determined  by  the  spectral  composition  of  the  luminance  and  the  intensity  of  the  energy 
of  an  object's  surface  reflection.  The  intensity  of  the  reflected  light  is  extremely 
inconsistent,  varying  due  to  factors  such  as  time-of-day  effects,  atmospheric  conditions, 
and  the  amount  of  airborne  particles  in  the  air.  This  lack  of  consistency  for  the  intensity 
of  reflected  light  required  the  HVS  to  develop  a  variety  of  mechanisms  to  disentangle  the 
contradictions  of  varying  lumination  and  thereby  to  achieve  nearly  constant  color 
perception  based  on  distal  surface  reflectivity  (Matlin  and  Foley,  1997).  A  major 
limitation  to  sensor  fusion  systems  is  that  these  mechanisms  cannot  be  duplicated  to 
achieve  the  same  constant  color  perception.  Sensor  fusion  systems  produce  color 
mappings  that  vary  with  changes  in  the  intensity  of  light's  spectral  composition.  The 
potential  disadvantage  caused  by  the  variability  in  spectral  illumination  may  be 
magnified  by  the  relatively  arbitrary  color  mappings  produced  by  this  new  technology. 
This  result  is  due  to  the  spectral  sensitivity  channels  that  compose  sensor  fusion 
technology  that  do  not  correspond  to  the  trichromatic  channels  of  the  HVS.  This  causes 
the  color  mapping  produced  by  fused  sensors  to  be  unnatural  in  appearance  (Krebs, 
1998).  Before  proceeding  with  a  discussion  on  previous  natural  and  false  color  studies,  a 
more  detailed  description  of  color  vision  may  aid  in  understanding  the  difficulties  in 
relating  false  color  to  natural  color. 

E.  HUMAN  VISUAL  SYSTEM 

The  HVS  is  an  extremely  complicated  system.  It  is  made  up  of  two  separate  but 
inter-related  systems;  the  eye  (receptor)  and  the  optic  neural  pathways  (processors) 
(Legrand,  1968).  The  sensory  receptors  in  the  eye  are  sensitive  to  energy  within  a  limited 
range  of  the  electromagnetic  spectrum,  specifically,  wavelengths  from  approximately  0.4 
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to  0.7  microns.  Figure  6  depicts  the  electromagnetic  spectrum.  Any  energy  beyond  this 
range  will  not  be  visually  detected  because  it  has  no  effect  on  the  receptors  (Sekular  and 
Blake,  1994). 
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Figure  6:  The  electromagnetic  spectrum.  (Matlin  and  Foley,  1997) 


Visual  processing  is  accomplished  by  a  system  of  neurological  pathways  between 
the  retina  and  the  brain  that  convert  the  reflected  light  energy  into  images.  A  more 
detailed  description  can  be  found  in  textbooks  on  vision  (Sekular  and  Blake,  1994). 

The  retina  is  the  interface  between  perception  and  reception  in  the  eye.  It 
contains  two  major  types  of  photoreceptor  cells  called  rods  and  cones.  Rods  are 


primarily  involved  in  vision  under  very  low  levels  of  light.  They  are  slow  acting,  provide 


minimal  visual  clarity,  and  cannot  discern  color.  Rod  vision  is  also  referred  to  as  scoptic 


vision  (Sheppard,  1968).  There  are  three  sub-types  of  cone  cells  that  are  fast  acting, 
sense  the  entire  visual  color  spectrum,  and  provide  maximum  clarity,  but  function  poorly 
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during  low  light  conditions.  Cone  vision  is  also  referred  to  as  photopic  vision  (Legrand, 
1968). 

Each  photoreceptor  cell  is  coated  with  a  chemical  substance  called  photopigments 
that  accomplish  the  transduction  of  light.  For  the  three  sub-types  of  cone  cells,  each 
photopigment  absorbs  more  light  in  one  portion  of  the  electromagnetic  spectrum  than  in 
any  other  area  (Matlin  and  Foley,  1997).  Estimates  of  the  portions  of  the  electromagnetic 
spectrum  that  are  absorbed  by  the  cone  cells  are  shown  in  Figure  7. 


Figure  7:  Spectral  Frequencies  of  the  L,  M,  S  cones.  (Wandell,  1995) 

As  seen  in  Figure  7,  each  particular  cone  cell  is  referred  to  as  the  S-cones 
(shortwave),  M-cones  (mediumwave)  and  L-cones  (longwave)  and  each  corresponds  to  a 
particular  color  with  the  color  spectrum,  blue,  green  and  red,  respectively  (Wandall, 
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1995). 


The  spectral  properties  of  light  are  measured  in  terms  of  the  amount  of  energy 
emitted  at  each  wavelength,  namely  the  spectral  power  distribution  of  the  light  source. 
The  portion  of  light  that  is  reflected  off  an  object's  surface  defines  the  surface-reflectance 
function.  This  reflected  energy  leads  to  different  amounts  of  absorption  in  the  three  cone 
classes  and  it  is  this  interpretation  of  cone  absorption  that  is  the  basis  of  color  perception. 
Therefore,  the  light  incident  to  the  HVS  is  a  function  of  the  spectral  power  distribution  of 
the  ambient  illumination  and  the  surface-reflectance  function  (Wandall,  1995).  For 
example,  suppose  that  the  eye  was  presented  a  target  color  (orange)  of  600-nanometer 
wavelength.  From  Figure  7,  the  target  would  present  a  large  amount  of  absorption  in 
both  the  M-cones  and  L-cones  and  the  color  would  consist  of  a  mixture  of  L-cones  (red) 
and  a  lesser  amount  of  M-cones  (green)  that  produces  the  color  orange.  The  remaining 
portions  of  the  reflected  energy  are  absorbed  by  the  object,  prohibiting  the  receptors  from 
receiving  energy  within  the  specific  frequency  bands  (Matlin  and  Foley,  1997). 

Isaac  Newton  demonstrated  in  the  late  17th  century  that  white  light  consists  of  a 
combination  of  different  colored  lights  that  are  evenly  distributed  across  the  visible 
electromagnetic  frequency  spectrum  (Matlin  and  Foley,  1997).  Figure  8  shows  the 
primary  colors  and  associated  wavelengths  in  relation  to  the  electromagnetic  spectrum. 
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Figure  8:  Visible  Color  Spectrum  (Matlin  and  Foley,  1997) 


It  is  difficult  to  imagine  that  objects  may  look  as  though  they  are  colored  when  in 
fact,  they  are  actually  reflecting  light  from  the  unabsorbed  portions  of  the  spectrum.  A 
chair  may  appear  blue,  but  in  reality  it  is  blue  because  its  surface  is  absorbing  most  of  the 
medium  and  long  wavelengths  and  is  primarily  reflecting  the  wavelengths  from  the  blue 
portion  of  the  spectrum  to  the  HVS.  Since  the  S-cones  would  absorb  the  majority  of  the 
energy  and  the  L-cones  and  M-cones  absorption  rates  may  be  negligible,  the  object  would 
be  perceived  as  blue  in  color  (Matlin  and  Foley,  1997). 

Another  characteristic  of  color  is  color  constancy.  In  color  constancy,  the  HVS 
tends  to  see  the  same  colors  during  varying  levels  of  illumination.  This  is  because  as  the 
illumination  changes,  the  sensitivity  distribution  of  a  light  source  will  change  at  a  similar 
rate  across  the  visible  frequency  spectrum.  The  distribution  of  wavelengths  reaching  the 
photopigments  of  the  cones  will  receive  the  same  information,  except  at  a  different 
intensity.  This  explains  why  grass  appears  to  be  green  on  a  bright  sunny  day  and  on  a 
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cloudy  rainy  day  (Sekular  and  Blake,  1994). 

Light  is  organized  by  its  wavelength  characteristics.  As  Isaac  Newton 
demonstrated,  white  light  that  enters  a  prism  is  broken  into  numerous  color  components. 
A  method  of  organizing  all  the  wavelengths  of  the  visible  spectrum  is  accomplished  by 
placing  them  in  a  circular  position,  or  color  wheel.  Figure  9  shows  the  standard  color 
wheel  and  distribution  of  frequencies  (Matlin  and  Foley,  1997). 


Figure  9:  Standard  color  wheel  and  associated  color  wavelengths. 

(Matlin  and  Foley,  1995) 

Color  is  also  characterized  by  the  amount  of  saturation  that  is  present.  A  light  that 
occupies  a  very  narrow  frequency  band  is  said  to  be  pure  and  highly  saturated.  An 
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example  would  be  that  of  a  laser  beam.  Saturation  measures  the  purity  of  color.  As  the 
width  of  the  frequency  band  of  the  light  expands,  additional  variants  of  color  are  added  as 
they  are  received  through  the  respective  cone  cells  and  the  representation  is  not  as  pure. 
Saturation  can  also  be  measured  on  the  color  wheel.  A  totally  saturated  color  is  displayed 
on  the  outside  edge  of  the  color  wheel  and  unsaturated  color  is  displayed  as  the 
wavelength  approaches  the  color  wheel  origin.  Saturation  is  only  possible  when  the 
brightness  is  intermediate,  neither  too  light  nor  too  dark  (Wandall,  1995).  Now  that  an 
overview  of  the  characteristics  of  color  and  the  HVS  has  been  presented,  we  can  now 
discuss  color's  role  in  object  recognition. 

F.  REVIEW  OF  THE  LITERATURE 

The  role  of  color  and  its  impact  on  the  human  perceptual  system  has  intrigued 
vision  researchers  for  many  years.  Until  recently,  color  has  been  attributed  to  object 
recognition,  in  that  it  was  assumed  that  color  aided  an  individual’s  ability  to  recognize 
objects  (Wurm,  et  al.,  1993).  Early  studies  suggest  that  color  vision  evolved  in  early 
humans  to  facilitate  the  search  and  recognition  of  food  (Polyak,  1957).  However,  it  has 
since  been  shown  that  individuals  who  possess  a  color  deficiency  (color  blindness)  only 
become  aware  of  this  deficiency  after  taking  a  color  test  (Steward  and  Cole,  1989). 
These  individuals  lead  a  normal  life  and  function  without  visual  impairment.  This  leads 
us  to  question  the  function  of  color  vision,  and  the  role  it  plays  in  object  recognition. 
Although  no  firm  conclusion  exists  concerning  the  role  of  color  in  object  recognition, 
most  vision  researchers  agree  that  color  is  significant. 

Three  tasks  have  typically  been  used  to  determine  colors  role  in  object 
recognition.  These  tasks  are  classification,  verification  and  naming.  Research  has  shown 
that  although  color  does  not  influence  classification  or  verification  performance,  it  does 


influence  naming  performance  (Joseph  and  Proffitt,  1996).  However,  it  will  be  later 
shown,  that  this  statement  is  often  debated.  For  purposes  of  this  thesis,  classification  is 
defined  as  objects  that  belong  to  a  specific  category,  such  as  objects  that  are  either  living 
or  non-living  (Davidoff  and  Ostergaard,  1988).  In  verification  tasks,  individuals  are 
tasked  to  validate  the  relationship  between  a  word  and  a  picture.  If  an  image  of  an  apple 
was  presented  to  a  participant,  the  participant  must  respond  whether  the  associated  word 
or  phrase  matched  the  presented  object  (Biederman  and  Ju,  1988). 

Biederman  and  Ju  (1988),  conducted  a  significant  study  on  the  role  of  color  in 
object  recognition.  This  study  compared  colored  pictures  of  manmade  objects  to  black 
and  white  line  drawings  of  the  same  objects.  The  objects  and  line  drawings  were 
presented  in  two  kinds  of  tasks.  The  first  task  was  a  naming  task.  Participants  saw  a 
slide  of  an  object  that  was  presented  in  either  a  color  photograph  or  line  drawing 
depiction.  Upon  recognizing  the  object,  participants  named  the  object  using  a  voice  key 
that  recorded  reaction  times  and  error  rates.  It  was  expected  that  participant  reaction 
times  and  error  rates  for  the  naming  task  would  be  smaller  for  objects  depicted  in  color 
format  (Biederman  and  Ju,  1988).  The  other  task  was  a  verification  task.  Participants 
had  to  verify  the  name  of  an  object  by  matching  it  to  the  image  representation.  This  was 
accomplished  by  pressing  a  YES  or  NO  response  on  a  microswitch.  Each  presentation 
was  shown  in  one  of  three  different  exposure  times  followed  by  a  mask.  The  exposure 
times  were  50, 65  and  100  milliseconds  in  duration.  It  was  assumed  that  longer  exposure 
duration  and  slightly  dimmer  projector  intensity  would  favor  colored  photography 
(Biederman  and  Ju,  1988). 

The  results  of  the  experiments  were  surprising.  In  the  naming  tasks  in  which  the 
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projector  intensity  was  varied  and  a  mask  that  immediately  followed  an  image 
presentation  was  presented  randomly,  no  significant  advantages  for  participant  reaction 
times  and  error  rates  for  the  color  images  were  shown.  An  astonishing  finding  during  the 
naming  task  portion  of  the  experiment  was  that  reaction  times  were  generally  larger  for 
images  displayed  in  line  drawings  when  observed  with  smaller  exposure  times.  This 
suggests  that  the  HVS  may  respond  less  efficiently  to  colored  images  when  the  images 
are  presented  momentarily  (Biederman  and  Ju,  1988). 

The  same  color  photographs  and  line  drawings  were  used  in  the  verification  task 
experiment.  Participants  were  given  a  verbal  one-word  description  of  a  target  and  were 
then  instructed  to  respond  once  an  image  was  presented.  This  task  enabled  the 
participants  to  form  a  mental  representation  of  the  target  and  to  also  anticipate  its  texture 
and  details.  The  results  of  the  verification  tasks  mirrored  the  results  of  the  naming  task. 
No  advantage  for  identifying  a  colored  picture  over  a  similar  black  and  white  outline  was 
found  (Biederman  and  Ju,  1988). 

The  researchers  concluded  that  simple  line  drawings  can  be  identified  as  quickly 
and  accurately  as  fully  detailed,  textured,  colored  pictures  of  the  same  object.  These 
results  support  the  premise  that  initial  access  to  a  mental  representation  of  an  object  can 
be  modeled  by  matching  the  edge-based  depiction  of  the  object.  Therefore,  when  an 
object's  edge  can  be  perceived  by  the  HVS,  color  plays  only  a  secondary  role  in  the  real¬ 
time  recognition  of  the  object  (Biederman  and  Ju,  1988). 

To  parallel  Biederman  and  Ju’s  research,  another  group  of  visual  scientists 
conducted  a  similar  experiment  investigating  the  role  of  color  in  categorical  judgements. 
Line  drawings  were  also  used  for  the  experiments.  Colored  and  black-and-white 
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transparencies  were  produced  for  each  object  that  was  shown  to  the  participants.  The 
objects  were  rated  according  to  size  (small,  smaller,  large,  and  larger)  compared  to  a 
trumpet,  which  was  defined  as  the  size  standard.  Additionally,  objects  were  categorized 
into  living  and  non-living  categories.  Participants  were  tasked  to  verbally  indicate  the 
specific  categories  in  which  the  objects  belonged.  For  example,  a  color  presentation  of 
an  airplane  would  be  placed  in  the  "much  larger,  non-living"  category.  Assuming  that 
color  would  have  no  influence  on  participant  reaction  time  in  the  task,  the  researchers 
were  interested  in  investigating  color  benefits  on  image  size  and  recognition  categories. 
For  example,  would  a  participant  recognize  a  colored  depiction  of  a  butterfly  or  an 
airplane  faster  when  compared  to  the  same  pictorial  representation  in  achromatic  format 
(Davidoff  and  Ostergaard,  1988)? 

The  results  of  the  experiment  showed  that  color  affected  neither  size  nor 
living/non-living  classifications.  Although  color  reduced  participant  reaction  times  when 
compared  to  reaction  times  of  monochrome  images,  the  results  were  not  significant.  The 
authors  speculated  that  the  slightly  smaller  reaction  times  for  the  colored  pictures  might 
have  been  attributed  to  a  color  brightness  advantage  (Davidoff  and  Ostergaard,  1988). 

In  a  subsequent  experiment,  identical  images  were  used  and  participants  were 
tasked  to  name  the  viewed  objects  instead  of  rating  them  for  object  size.  This  procedure 
attempted  to  determine  the  relationship  of  color  and  naming  living  and  non-living 
objects.  Non-living  objects  typically  are  not  normally  associated  with  a  particular  color 
and  therefore  it  was  hypothesized  that  the  color  advantage  would  enhance  living  object 
recognition  better  than  non-living  object  recognition.  Previous  studies  (Davidoff  and 
Ostergaard,  1985)  indicated  that  color  may  be  beneficial  in  object  naming  but  not  in 
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object  recognition.  Their  findings  however,  were  based  on  an  inadequate  image  sample 
size.  Therefore,  the  focus  of  this  naming  task  was  to  validate  previous  findings  with 
large  image  sample  sizes.  The  findings  of  the  research  indicated  that  color  did  not 
significantly  enhance  participant  reaction  time  between  living  and  non-living  conditions. 
It  did  have  a  positive  effect  on  participant's  reaction  times  and  supported  the  previous 
object  recognition  research  but  the  advantage  was  found  to  be  not  significant.  It  was 
concluded  that  color  played  a  role  in  object  naming  but  has  no  effect  in  object  recognition 
(Davidoff  and  Ostergaard,  1988).  This  conclusion  contradicts  Biederman  and  Ju's  studies 
that  used  similar  procedures,  but  different  types  of  pictures.  A  factor  not  controlled  and  a 
possible  contention  for  these  inconsistent  results  was  speculated  to  be  the  failure  of 
Biederman  and  Ju  to  control  luminance  characteristics.  It  was  suggested  that  visual 
analysis  of  the  color  images  may  have  been  at  a  starting  disadvantage  because  of  greater 
limitations  in  edge  extraction  compared  to  the  line  drawings  (Wurm,  et  al.,  1993). 

Price  and  Humphries  (1989)  examined  the  effects  of  color  congruency  and 
photographic  detail  on  naming  and  classification  of  structurally  similar  and  dissimilar 
objects.  Using  the  notion  that  color  does  not  influence  object  recognition  with  edge- 
based  effects  (Biederman  and  Ju,  1988),  they  wanted  to  determine  if  these  theories  could 
be  applied  to  surface-based  effects.  Surface-based  effects  predict  that  it  will  benefit 
object  recognition  if  the  effects  are  shown  in  conjunction  with  their  surface  details,  such 
as  a  variation  in  luminance,  brightness,  texture  and  color.  To  determine  these  effects,  the 
authors  conducted  a  series  of  experiments  in  which  participants  performed  subordinate 
naming  and  classification  tasks  using  structurally  similar  and  dissimilar  objects.  Items 
such  as  birds  were  observed  with  structurally  similar  shapes  and  tools  for  instance,  were 
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presented  with  structurally  dissimilar  shapes.  Price  and  Humphries  hypothesized  that 
object  color  and  surface  detail  would  be  beneficial  for  discriminating  between  categorical 
members.  They  felt  that  classification  of  objects  could  proceed  directly  from  the 
structural  description  of  the  object.  Their  findings  supported  the  hypothesis  and  revealed 
that  surface  color  benefits  not  only  a  naming  task  but  also  applies  to  classification  tasks 
when  shape  information  is  not  sufficient  for  discriminating  between  category  members. 
They  also  suggested  that  surface  color  is  more  likely  to  be  important  for  specific 
categories  of  objects,  namely  natural  categories,  as  opposed  to  artifactual  categories.  It 
was  therefore  concluded  that  the  effects  of  color  were  stronger  for  categories  with 
structurally  similar  characteristics.  This  is  because  naming  an  object  requires  finer 
within-category  differentiation.  The  finer  the  required  differentiation,  the  stronger  the 
effects  of  surface  detail  (Price  and  Humphries,  1989). 

Wurm,  Legge,  Isenberg,  and  Luebker  (1993)  questioned  color's  role  in  object 
recognition  using  participants  who  had  normal  and  degraded  color  vision.  They  believed 
that  color  and  shape  information  interacted  in  object  recognition,  especially  when  spatial 
resolution  was  degraded.  They  also  hypothesized  that  color  would  improve  object 
recognition  for  individuals  with  degraded  visual  acuity.  Previous  findings  indicated  that 
color  had  no  practical  advantage  for  low  visual  reading,  but  the  lack  of  research  on  low 
vision  object  recognition  prevented  the  correlation  between  color's  involvement  in 
reading  with  object  recognition  (Wurm,  et  al.,  1993). 

The  authors  chose  food  items  as  objects  for  viewing.  This  was  because  using 
man-made  objects,  such  as  those  used  in  prior  research,  possessed  stereotypical  shapes. 
Food  objects  tend  to  have  a  wide  range  of  color  and  shape  representation.  For  example, 
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an  apple  may  be  red,  green  or  yellow,  and  may  be  whole,  sliced  or  bitten.  The  authors 
speculated  that  the  reason  for  the  inconsistent  findings  in  previous  recognition  studies 
was  that  luminance  was  not  controlled  between  the  chromatic  and  achromatic  imagery. 
The  authors  eliminated  any  effect  of  luminance  prior  to  conducting  the  experiment. 
Participants  were  shown  84  images  (21  items  in  four  different  poses)  in  one  of  four 
different  formats.  These  formats  were  both  blurred  color  and  monochrome  and  unblurred 
color  and  monochrome.  Using  a  naming  task,  participants  were  required  to  rapidly  name 
the  food  item  shown.  There  were  two  major  findings  from  the  results.  First,  color  did 
improve  the  recognition  of  food  objects  and  second,  an  interaction  between  color  and 
spatial  resolution  did  not  exist.  This  implies  that  color  appears  to  improve  object 
recognition  equally  for  both  blurred  and  unblurred  targets  (Wurm,  et  al.,  1993). 

Next,  the  authors  tested  the  vision  deficient  participants.  Each  participant  was 
categorized  according  to  specific  visual  impairment.  The  participants  were  tasked  to 
view  each  of  the  21  food  items  in  four  different  poses  for  a  total  of  84  different 
presentations.  The  images  were  magnified  by  a  factor  of  two  to  facilitate  viewing.  Each 
participant  equally  viewed  images  in  either  a  color  or  achromatic  representation.  The 
authors  hypothesized  that  color  would  benefit  participants  with  visual  deficiencies  in 
object  recognition  (Wurm,  et  al.,  1993). 

Although  participant  reaction  times  for  identifying  images  viewed  in  color  was 
half  that  of  images  viewed  in  monochrome,  statistical  analysis  failed  to  show 
significance.  These  results  were  probably  due  to  the  large  variability  of  the  data  due  to 
the  heterogeneity  of  the  different  visual  disorders.  The  authors  concluded  that  color 
enhances  object  recognition  for  people  with  low  color  vision,  but  object  recognition  is 
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hindered  by  poor  visual  acuity.  They  also  found  that  color  and  visual  acuity  acted 
independently  in  low  vision  object  recognition  (Wurm,  et  al.,  1993). 

Finally,  the  authors  attempted  to  determine  the  relationship  of  color  and 
prototypical  images.  Prototypical  images  were  defined  as  objects  that  possessed  a  typical 
color  and  shape.  Each  of  the  objects  was  ranked  for  prototypically.  It  must  be  clarified 
that  prototypical  objects  are  much  different  than  objects  with  diagnostic  colors.  While 
the  color  red  may  be  prototypically  associated  with  an  apple,  it  is  also  associated  with 
numerous  other  objects  such  as  tomatoes  and  cherries.  However,  orange  is  associated 
with  relatively  few  objects,  namely  carrots  and  pumpkins,  that  are  identified  by  their 
distinctive  profiles.  Two  factors  must  be  present  to  determine  when  color  is  diagnostic. 
First,  the  color  must  be  symptomatic  of  the  object  (i.e.  green  is  a  symptom  of  spinach), 
and  the  other  objects  presented  in  the  allowable  domain  should  not  be  of  this  color. 
Therefore,  since  other  foods  in  the  experiment  were  also  green,  green  was  not 
"diagnostic"  of  spinach  (Wurm,  et  al.,  1993).  It  was  expected  that  object  reaction  times 
would  be  smaller  for  the  most  prototypical  images.  This  was  because  prototypical 
images  may  be  recognized  faster  because  they  contain  certain  shape  characteristics  that 
are  stored  in  memory.  Since  less  prototypical  images  may  have  less  defined  shape 
attributes,  it  is  expected  that  the  advantage  of  color  increased  indirectly  with  prototypical 
images.  Participants  were  asked  to  estimate  probabilities  of  the  food  categories  given  a 
specific  color.  The  results  suggested  that  the  participant's  knowledge  about  the  color  of 
the  food  item  did  not  account  for  the  advantage  of  color  in  object  recognition.  To 
conclude  the  experiments  findings,  the  authors  found  that  color  improved  object 
recognition  but  the  mechanism  was  sensory  rather  than  cognitive  (Wurm,  et  al.,  1993). 
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Joseph  and  Proffitt  (1996)  conducted  a  series  of  experiments  to  determine 
semantic  versus  perceptual  influences  of  color  in  object  recognition.  They  theorized  that 
a  significant  reason  for  the  contradictory  results  of  previous  research  was  that  stored  and 
surface  color  knowledge  must  play  a  role  in  recognition  tasks.  Surface  knowledge  is 
defined  as  the  depicted  color  that  represents  the  surface  of  the  object,  such  as  the  color 
yellow  for  a  banana.  However,  surface  color  can  be  arbitrary  because  an  object  can  be 
represented  by  any  color.  Stored  color  knowledge  is  the  semantic  information  about  the 
prototypical  colors  of  objects,  such  as  the  knowledge  that  bananas  are  typically  yellow. 
Stored  color  knowledge  is  not  arbitrary  because  people  have  shared  ideas  about  the 
typical  colors  of  objects  (Joseph  and  Proffitt,  1996).  The  authors  felt  that  an  objects 
surface  color  must  be  compared  to  the  activated  prototypical  color  of  the  image  for 
recognition.  The  basis  for  their  research  was  to  determine  whether  the  decision  that  two 
objects  were  the  same  depends  on  the  activation  of  stored  color  knowledge  or  on  the 
processing  of  surface  color  (Joseph  and  Proffitt,  1996). 

Using  a  series  of  experiments  in  which  they  tested  the  effects  of  surface  color;  the 
effects  of  stored  knowledge;  and  the  effects  of  combined  surface  and  stored  knowledge  in 
verification  tasks,  they  found  that  congruent  color  knowledge  (red  apples)  made 
verification  easier  than  incongruent  (purple  apples)  surface  color.  Although  these  results 
conflicted  with  Biederman  and  Ju  (1988)  findings,  they  contributed  these  results  to 
objects  used  in  the  imagery.  It  was  assumed  that  natural  objects  have  stronger 
associations  with  colors  than  artifactual  objects.  Therefore,  the  use  of  surface  color  as  a 
cue  was  more  beneficial  for  object  recognition  from  natural  categories  (Joseph  and 
Proffitt,  1996).  They  also  found  that  stored  color  knowledge  enhanced  object 
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recognition.  When  both  image  and  word  stimuli  were  similar  by  shape  and  prototypical 
color,  more  interference  occurred  when  compared  to  stimuli  that  were  similar  by  only 
shape.  When  they  compared  stored  and  surface  color  information  for  object  recognition, 
they  found  that  stored  color  knowledge  had  a  more  profound  influence  than  surface  color 
knowledge  (Joseph  and  Proffitt,  1996). 

Obviously,  the  effects  that  contributed  to  the  inconsistent  results  were  primarily 
due  to  the  experimental  methodologies  used  to  conduct  each  task.  It  appears  that  color 
has  no  influence  when  viewing  artifactual  images  because  prototypical  effects  are  not 
pronounced.  Additionally,  presenting  a  label  prior  to  an  image  in  verification  tasks  may 
prime  the  participant  and  mask  the  surface  color  effects  (Joseph  and  Proffitt,  1996). 
Throughout  these  research  examples,  items  were  presented  in  singular  form  and 
displayed  against  a  neutral  backdrop  in  a  controlled  environment.  No  such  experiment 
compared  images  displayed  in  the  global  environment.  Therefore,  it  cannot  be  assumed 
that  a  participant  may  be  able  to  identify  an  apple  in  a  tree  faster  when  presented  in  color 
than  in  monochrome.  Knowing  that  natural  color  may  generally  enhance  object 
recognition,  the  same  theories  that  are  applied  to  pseudo-color  that  is  displayed  by  fused 
sensor  displays  may  be  erroneous.  Research  into  exploring  the  potential  benefits  of 
adding  color  to  a  scene  is  currently  ongoing  and  also  shows  inconsistent  results.  The 
following  is  a  brief  synopsis  of  some  research  studies  being  conducted  concerning  color 
fusion. 

Current  sensor  fusion  research  that  examines  the  potential  advantages  of  pseudo¬ 
color  over  monochrome  is  based  on  three  types  of  experiments.  These  types  are  paired 
comparisons,  reaction  time  and  mean  detection  accuracy,  and  the  accuracy  of  target 
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detection. 


In  the  paired  comparison  type  of  experiment,  participants  are  presented  a  random 
sequence  of  image  pairs  and  are  asked  to  respond  to  the  "better"  presentation.  The  goal 
of  this  type  of  experiment  is  to  show  differences  in  the  preferred  ordering  of  the  sensors 
and  to  deduce  if  fusion  based  sensor  imagery  is  more  preferred  than  an  individual  sensor 
display.  This  type  of  experiment  was  utilized  by  Krebs,  Buttrey,  Lewis  and  McKenzie 
(1997)  to  identify  a  preferred  sensor  for  use  in  a  low- vision  environment.  In  this 
experiment,  participants  were  shown  pair-wise  comparisons  of  25  different  scenes  using 
five  different  sensor  formats.  These  formats  were  image  intensified  (I2 ),  infra-red,  fused 
monochrome  and  two  different  fused  color  algorithms.  The  participant  was  tasked  to 
choose  a  preferred  format  while  locating  a  navigational  cue  contained  within  the  scene. 
The  results  of  the  experiment  were  inconclusive.  It  was  assumed  that  the  inconsistencies 
that  resulted  from  the  experiment  were  due  to  a  sensor  to  scene  interaction.  Sensors 
appeared  to  perform  differently  depending  on  the  presented  scene  characteristics  such  as 
the  texture,  color  and  content.  Although  a  slight  advantage  in  recognizing  objects  using 
the  color-fused  techniques  was  shown,  participant  performance  was  actually  very 
dependent  on  the  type  of  navigational  cues  used.  Results  across  all  conditions  were 
inconsistent  (Krebs,  Buttrey,  Lewis  and  McKenzie,  1997). 

Measurement  of  search  accuracy  and  reaction  times  appear  to  provide  a  better 
foundation  in  the  effort  to  determine  if  sensor  fusion  imagery  is  preferred  over 
achromatic  imagery.  Steele  and  Perconti  (1997)  conducted  a  series  of  tests  to  determine 
if  the  use  of  color  fusion  imagery  enhanced  the  performance  or  improved  the  situational 
awareness  of  helicopter  pilots  during  low-visibility  flights.  The  experiment  was  a  part- 
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time  simulation  and  identification  task  using  25  different  scenes  collected  from  the  sensor 
combinations  in  both  still  frame  and  video  format.  The  Advanced  Helicopter  Pilotage 
System  (AHPS)  was  used  to  collect  a  series  of  night  pilotage  maneuvers  using  various 
sensor  combinations.  This  imagery  was  then  used  to  determine  horizon  perception, 
object  recognition,  identification  and  geometric  perspective  tasks.  In  the  still-frame 
experiment,  scenes  were  randomly  presented  for  at  most  10  seconds.  Participant 
response  time  (YES/NO)  and  accuracy  data  was  collected  from  specific  information 
extracted  from  the  scene.  Object  recognition  and  identification  tasks  required  the 
participants  to  determine  if  a  specific  object  was  present  or  not.  These  tasks  also  required 
the  participant  to  locate  a  specific  object  and  determine  its  position  in  the  field  of  view  or 
to  describe  the  object  in  detail.  An  observer  was  also  required  to  determine  if  any  object 
existed  that  would  hinder  the  flight  while  the  aircraft  was  on  its  intended  flight  path. 
Horizon  perception  tasks  required  participants  to  determine  whether  the  horizon  was 
level  or  not.  The  geometric  perspective  tasks  required  the  participant  to  identify  the 
shape  or  orientation  of  a  feature  using  monocular  perception  cues  (Steele  and  Perconti, 
1997).  The  results  of  the  still-frame  experiments  showed  that  fused  imagery,  whether 
color  or  monochrome,  led  to  better  performance,  but  the  performance  was  highly 
dependent  on  the  particular  fusion  algorithm,  the  particular  visual  task,  and  the  scene 
content  (Steele  and  Perconti,  1997). 

The  second  experiment  utilized  short  (17  to  31  second)  video  clips  that  were 
collected  by  the  AHPS.  These  video  clips  were  presented  to  participants  in  one  of  five 
different  formats  that  tested  response  time  and  accuracy  while  identifying  a  specific 
object.  Following  each  video  sequence,  participants  were  asked  to  rate  the  imagery  based 
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on  the  five  point  Likert  scale.  The  results  failed  to  identify  a  preferred  individual  sensor 
format.  The  preference  of  a  sensor  format  was  scene  specific  (sensor  to  scene 
interaction)  and  that  no  particular  sensor  was  found  to  be  "desirable"  for  night  helicopter 
pilotage.  This  experiment  confirmed  the  previous  experimental  findings  that  sensor 
fusion  is  highly  dependent  on  the  algorithm  used,  as  well  as  task  and  scene  content 
(Steele  and  Perconti,  1997). 

Toet,  Ijspeert,  Waxman  and  Aquilar  (1997)  conducted  a  study  to  investigate 
whether  increasing  scene  detail  with  sensor  fusion  would  improve  a  participants  ability  to 
perform  a  situational  awareness  search  task.  Images  were  collected  using  both  visible 
and  infra-red  sensors  that  consisted  of  scenes  taken  at  dawn  when  low  light  conditions 
and  minimal  thermal  contrast  existed.  During  the  image  collection  process,  a  target  was 
presented  that  contained  a  high  infra-red  contrast.  The  images  were  fused  into  four 
different  formats  consisting  of  color  and  monochrome.  Participants  were  instructed  to 
identify  location  and  error  distances.  The  data  for  the  incorrect  responses  was  then 
calculated.  The  results  showed  that  participants  performed  better  using  fused  color 
images  when  compared  to  fused  monochrome  and  individual  sensor  images  (Toet,  et  al., 
1997). 

These  inconsistent  sensor  fusion  studies  may  be  attributed  to  the  differences  in 
fusion  algorithms  used  in  the  various  experiments  and  to  the  differences  in  the  employed 
psychophysical  tasks.  A  reason  for  the  inconsistent  results  may  result  because  fused 
imagery  does  not  always  improve  participant  performance  when  compared  to  individual 
sensor  imagery.  There  may  exist  certain  conditions  when  the  fusion  process  aids 
performance  and  other  situations  when  fusion  is  not  necessary.  If  this  were  true,  the 
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experiments  should  be  designed  that  precisely  elucidate  when  and  where  the  fusion 
process  is  helpful.  A  more  probable  explanation  and  one  that  is  easy  to  control  is  that 
proper  experimental  methodology  for  adequately  testing  these  images  has  not  yet  been 
employed.  These  previous  studies  contain  potential  experimental  methodological 
problems  that  may  hinder  accurate  evaluation  of  the  various  image  processes. 

G.  HYPOTHESIS 

A  significant  deficiency  in  color  fusion  research  is  that  researchers  have 
attempted  to  utilize  the  findings  from  natural  and  achromatic  studies  and  correlate  the 
principles  to  the  false-color  theories  that  are  used  in  sensor  fusion.  The  majority  of  the 
color  studies  were  conducted  in  the  laboratory  environment,  utilizing  single,  defined 
natural  or  manmade  objects  observed  in  an  unnatural  setting.  Comparing  the  findings 
from  these  types  of  experiments  to  research  being  conducted  with  natural  scenes  and  a 
mixture  of  living  and  non-living  objects  is  a  significant  threat  to  external  validity.  No 
known  studies  have  been  found  that  compare  HVS  perception  of  a  global  scene  with  the 
same  scene  observed  with  imagery  obtained  from  a  fused  color  sensor. 

The  experimental  complexity  of  sensor  fusion  studies  has  proven  to  be 
overwhelming  in  the  search  for  verification  that  it  provides  a  better,  more  appealing  scene 
for  the  observer.  Efforts  have  generally  focused  on  the  end  product  of  object  recognition 
and  have  only  shown  that  the  performance  of  fusing  combinations  of  night  sensors  varies 
with  the  algorithms  used  and  scene  condition.  A  significant  deficiency  to  sensor  fusion  is 
that  in  some  environmental  conditions,  observers  have  exhibited  better  performance  with 
individual  sensor  imagery.  The  research  conducted  espouses  the  superiority  of  fused 
imagery  by  using  specific  environmental  conditions  or  subjective  operator  opinion.  In 
light  of  these  inconsistencies,  this  thesis  will  attempt  to  relate  individual  preferences 
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between  identical  scenes  manipulated  with  various  color,  pseudo-color  and  achromatic 
formats. 

Natural  scenes  have  been  carefully  selected  by  pre-designated  standards  taken 
from  a  commercially  available  collection  of  digital  photographs  on  a  Softkey  Photo 
Gallery  CD  published  by  the  Learning  Company.  Some  of  the  categories  of  images  were 
scenes  from  the  states  of  Washington,  Utah,  and  Colorado;  farming  scenes,  and  pictures 
of  animals;  and  pictures  of  sporting  events.  Each  sensor  was  identically  manipulated  so 
that  the  only  difference  a  participant  would  see  was  the  particular  color  or 
monochromatic  format.  Reducing  the  variability  of  processing  with  same  scene  displays 
controls  the  cognitive  process.  The  feature  maps  associated  with  an  image  and  the  effects 
of  sensor  fusion  can  then  be  determined.  The  feature  maps  are  dependent  on  the  overall 
scene  context.  Sufficient  scenes  have  been  selected  to  ensure  the  experiments  look  across 
multiple  real  world  scene  environments  and  not  just  those  that  have  shown  superior 
fusion  performance. 

Two  experiments  were  conducted  to  measure  subject  accuracy  (error  rates  in 
percent)  and  reaction  time  (in  milliseconds)  by  determining  the  orientation  of  natural 
color,  artificial  color,  and  achromatic  scenes.  The  purpose  of  the  experiment  is  to 
determine  which  type  of  scene  is  preferred  between  natural  color,  false  color  or 
monochrome,  and  to  identify  specific  deficiencies  that  may  cause  false  color  scenes  to  be 
processed  less  efficiently. 

The  second  experiment  will  be  identical  to  the  first  experiment  except  Gaussian 
noise  will  be  substituted  for  false  saturation.  This  experiment  will  compare  the  findings 
of  the  previous  experiment  under  ideal  conditions  with  the  findings  under  conditions  of 
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low  resolution.  This  may  add  validity  to  previous  findings  that  color  is  useful  to  help 
segment  object  shapes  in  the  absence  of  defined  edge  information.  Additionally,  this 
experiment  will  closely  match  true  representations  of  imagery  displayed  to  a  user  when 
using  a  fused  device  under  realistic  conditions. 

In  lieu  of  the  previous  studies,  it  is  expected  that  reaction  times  and  accuracy  for 
false  color  scenes  will  be  slower  and  less  accurate  when  compared  to  a  natural  color  or 
achromatic  scene.  Under  degraded  conditions,  it  is  hypothesized  that  the  effect  will  be 
more  severe,  that  natural  color  imagery  will  be  highly  preferred  compared  to  a  false-color 
and  achromatic  scene.  A  level  of  significance  of  a  =  0.5  will  be  used  for  all  statistical 
purposes. 


41 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


42 


III.  EXPERIMENT  I 


A.  METHODS 

1.  Participants 

Twenty  students  from  various  military  services  and  job  specialties  undergoing 
academic  studies  at  the  Naval  Postgraduate  School  or  at  the  Defense  Language  Institute 
volunteered  for  this  study.  All  participants,  by  self-report,  possessed  normal  color  vision 
and  had  at  least  20/20  corrected  vision.  Participants  were  uninformed  concerning  the 
purpose  of  the  experiment.  Before  participating  in  the  experiment,  participants  were 
granted  informed  consent. 

2.  Apparatus 

The  experimental  workstation  consisted  of  a  200  MHZ  Pentium  personal 
computer  equipped  with  a  Texas  Instrument  TMS-340  Video  Board  and  the 
corresponding  TIGA  Interface  to  Vision  Research  Graphics  software.  Stimuli  were 
presented  on  an  IDEK  MF-8521  High  Resolution  color  monitor  (21”  X  20”  viewable 
area)  equipped  with  an  anti-reflect,  non-glare,  P-22  short  persistence  CRT.  Pixel  size 
was  .26”  horizontal  and  .28”  vertical;  the  resolution  was  800  X  600  square  pixels  and  the 
frame  rate  was  98.7  Hz.  The  brightness  of  the  monitor  was  linearized  by  means  of  an 
eight-bit  color  look-up  table  (LUT)  for  the  red,  blue  and  green  guns.  Responses  were 
recorded  on  the  number  pad  of  a  standard  (IBM  clone)  keyboard.  Moderate  ambient 
luminance  (1.2  cd/m 2  luminance)  was  maintained  using  a  small  floor  lamp  placed  on  the 
floor  behind  the  IDEK  monitor.  Viewing  distance  was  approximately  150  centimeters 
and  the  subjects  were  free  to  move  their  heads.  A  chair  was  provided  for  subject  comfort 
and  to  help  maintain  the  appropriate  distance  and  viewing  angle.  Overhead  lights  were 
extinguished  and  subjects  were  required  to  adapt  to  the  darkness  for  approximately  two 
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minutes  prior  to  conducting  the  experiment. 

3.  Stimuli 

Each  of  270  images  (30  of  which  were  designated  for  practice)  was  presented  in 
one  of  eight  different  formats.  These  formats  were  natural  color,  color  with  false  hue, 
saturation,  or  both  hue  and  saturation  manipulated,  natural  gray  and  gray  with  false  hue, 
saturation,  or  both  hue  and  saturation  manipulated.  Images  were  also  presented  in  an 
upright  or  inverted  position,  so  that  each  image  was  presented  in  16  possible  ways. 

Construction  of  the  experiment  stimuli  began  by  choosing  images  under  specific 
pre-arranged  criteria  determined  by  two  vision  science  experts.  The  criterion  for 
selection  was  that  a  horizon  was  not  present  to  contribute  to  identifying  orientation. 
Images  that  contained  reflections  and  shadows  were  eliminated  because  they  may 
confuse  the  participant.  All  image  manipulations  were  performed  using  Adobe 
Photoshop  Version  4.0  Illustrator  Software.  The  images  were  first  cropped  to  a  square 
400  X  400  pixel  size.  Each  of  the  270  images  was  then  manipulated  within  the 
Photoshop  software  using  the  standard  image  processing  functions. 

The  hue  of  the  images  was  manipulated  by  adjusting  the  hue  value  of  each  pixel 
and  replacing  it  by  its  complementary  color.  The  concept  of  complementary  colors  arises 
from  a  modem  version  of  Newton’s  color  wheel.  The  color  wheel  organizes  all  the  colors 
according  to  the  visible  light  spectmm  wavelengths.  The  exception  to  the  color  wheel 
organization  is  the  region  that  is  complementary  to  the  green  region.  This  region  is  called 
the  nonspectral  hue.  It  is  produced  by  various  combinations  of  the  other  colors.  The 
color  magenta  is  a  combination  of  the  blue  and  red  colors  (Figure  10).  The  remainder  of 
the  colors  on  the  color  wheel  represent  only  the  monochromatic  colors  -  those  colors  that 
are  produced  by  a  single  wavelength  (Matlin  and  Foley,  1997). 
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The  complement  of  a  color  is  found  by  drawing  a  straight  line  from  a  particular 
color,  through  the  wheel  center,  to  the  intersection  on  the  opposite  side  of  the  wheel.  For 
example  from  Figure  10,  the  complementary  color  of  yellow  is  blue.  It  was  in  this 
manner  that  the  color  of  the  images  in  the  two  experiments  were  changed. 

There  are  two  different  ways  in  which  color  is  mixed.  Additive  mixture  of  colors 
infers  that  particular  light  beams  are  combined,  not  the  color  pigments  from  the 
respective  parts  of  the  spectrum.  Additive  mixing  of  complementary  colors,  such  as 
mixing  blue  and  yellow  in  equal  portions,  produces  gray.  The  subtractive  mixture  of 
colors  infers  that  the  color  pigments  are  mixed,  or  that  two  or  more  colored  filters  are 
placed  together  and  a  light  beam  is  emitted  through  these  filters.  The  subtractive  mixing 
of  blue  and  yellow  produces  green  (Matlin  and  Foley,  1997). 


Figure  10:  Illustration  of  the  modern  color  wheel. 
Courtesy  of  http://www.ricklineback.com/term4.htm 


The  Photoshop  software  provides  a  slide  bar  with  a  range  from  -180  degrees 
(counterclockwise  from  0)  to  +180  degrees  (clockwise  from  0)  for  hue  manipulation. 
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Rotating  along  the  color  wheel  —180  or  +180  degrees  results  in  the  exact  same  outcome; 
the  original  image's  color  complement.  For  consistency,  all  images  were  reassigned  a 
value  of  +180  degrees.  Figure  11  shows  an  original  image  and  the  image's  color 
complement  (image  manipulate  for  hue).  As  seen,  altering  an  image's  hue  gives  it  an 
unnatural  appearance. 


Figure  11:  Natural  colored  image  and  image  manipulated  for  Hue. 


Courtesy  of  Softkey  Photo  Gallery  CD  published  by  the  Learning  Company 

The  image's  saturation  was  manipulated  by  taking  each  color  pixel  and  fully 
saturating  the  color.  Colors  high  in  purity  (saturation)  are  arranged  around  the  edge  of 
the  color  wheel.  As  the  saturation  level  moves  toward  the  center  of  the  color  wheel,  the 
colors  become  increasingly  less  pure  and  more  faded,  moving  toward  an  evenly  balanced 
mixture  of  wavelengths  with  no  single  wavelength  dominance  (Sekular  and  Blake,  1994). 
Referring  to  the  color  wheel  in  Figure  12,  saturation  is  seen  as  the  apparent  purity  of  a 
color. 
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Figure  12:  Color  wheel  plus  Saturation.  Notice  how  the 
color  blue  becomes  lighter  as  it  moves  toward  the  center  of 
the  color  wheel.  (Matlin  and  Foley,  1997) 

Photoshop  software  provides  a  slide  bar  with  a  range  from  -100  to  +100.  An 

image  reassigned  a  value  of  +100  is  considered  to  be  totally  saturated.  An  image 
reassigned  a  value  of  -100  is  considered  to  be  achromatic.  All  images  were  reassigned  a 
value  of  +100.  Figure  13  shows  an  original  image  and  the  image  manipulated  by 
changing  its  saturation.  For  example,  a  saturated  blue,  one  lying  on  the  color  wheel  edge, 
is  perceived  as  deep  hue;  an  unsaturated  blue,  close  to  the  color  wheel  origin  at  the  same 
hue,  is  perceived  as  a  very  light  blue  or  baby  blue.  As  seen,  altering  an  image’s  saturation 
will  not  adversely  change  the  color,  it  only  changes  the  brightness  of  the  image. 
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Figure  13:  Natural  colored  image  and  image  manipulated  for  Saturation. 
Courtesy  of  Softkey  Photo  Gallery  CD  published  by  the  Learning  Company 


To  manipulate  an  image  for  both  hue  and  saturation,  the  values  for  the  images 
were  reassigned  by  the  Photoshop  software  a  hue  value  of  +180  degrees  and  the 
saturation  value  of +100.  Figure  14  provides  a  visual  representation  of  the  original  image 


and  the  image  with  both  the  hue  and  saturation  manipulated. 


Figure  14:  Natural  colored  image  and  image  manipulated  for  Hue  and  Saturation. 
Courtesy  of  Softkey  Photo  Gallery  CD  published  by  the  Learning  Company 


Achromatic  images  were  first  manipulated  in  the  same  fashion  as  the  natural 
colored  images,  then  converted  to  grayscale.  The  purpose  of  the  grayscale  images  was  to 
provide  a  control  for  changes  in  luminance  that  might  accompany  manipulations  of  color. 
Each  image  representation  is  seen  in  Figure  15. 
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Figure  15:  Achromatic  images:  A)  natural  gray,  B)  gray  with  Hue  manipulated,  C) 
gray  with  Saturation  manipulated,  D)  gray  with  Hue  and  Saturation  manipulated. 

Courtesy  of  Softkey  Photo  Gallery  CD  published  by  the  Learning  Company 

4.  Procedure 

To  prevent  learning,  each  subject  participated  in  one  but  not  both  experiments. 
Additionally,  each  image  was  viewed  once  in  one  of  the  various  formats.  Each  subject 
was  thoroughly  briefed  only  on  the  background  and  procedures  of  the  experiment  and 
was  given  the  opportunity  to  ask  questions.  Subjects  were  instructed  to  indicate  as 
rapidly  and  accurately  as  possible  with  a  keypress,  whether  the  image  orientation  was 
upright  (by  pressing  1)  or  inverted  (by  pressing  2).  Each  image  remained  on  the  screen 
until  the  subject  responded.  A  computer  generated  audio  tone  provided  feedback 
following  each  incorrect  response.  Subsequent  images  were  presented  following  a 
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response  after  a  slight  delay.  After  30  consecutive  image  presentations,  the  experiment 
paused  to  allow  the  subject  an  opportunity  to  relax.  The  experiment  restarted  when  the 
subject  pressed  any  key.  The  entire  experiment  lasted  approximately  20  minutes. 

Each  subject  was  given  a  block  of  thirty  practice  trials  prior  to  the  experiment. 
The  practice  trials  were  presented  in  the  same  format  as  the  experiment.  Upon 
completion,  subjects  were  offered  a  brief  pause  to  ask  any  questions.  The  actual 
experiment  then  commenced  and  the  participants  were  presented  with  240  images. 
Reaction  times  and  accuracy  were  collected  for  each  trial. 

5.  Experimental  Design 

The  design  was  a2X2X2X2  within-subjects  factorial  design.  Factors 
included  orientation  (upright  or  inverted),  color  (color  or  gray),  hue  (natural  or  false)  and 
saturation  (natural  or  false).  Each  of  the  16  cells  contained  15  observations  for  a  total  of 
240  data  points.  Reaction  times  for  incorrect  responses  and  responses  to  the  30  practice 
images  were  not  included  in  the  data  analysis.  There  were  no  theoretical  expectations  of 
the  colors  effect  on  inverted  images  from  prior  research,  therefore,  the  reaction  times  to 
inverted  images  were  also  discarded.  However,  inverted  error  rates  were  recorded  and 
viewed  to  ensure  participants  did  not  display  any  bias  toward  a  particular  orientation 
selection. 

B.  DEVELOPMENT  OF  THE  VISUAL  MODEL 

As  with  any  psychophysical  experiment,  inconsistencies  occur  between  each 

subject’s  response  times  and  error  rates.  These  differences  arise  in  part  due  to  variances 
in  individual  participants,  image  conditions  and  manipulation.  To  account  for  these 
differences,  the  following  model  is  proposed: 


50 


Reaction  TimeyWmn  =Part(.  +  Color .  +  Hue*  +  Sat,  +  Image  m  +  error  n 

where  Part,,  represents  the  ith  participant  of  the  experiment; 

Color  .  represents  the  color  of  the  image  (natural  or  monochrome); 

Hue*  represents  the  hue  condition  of  the  image  (natural  or  false); 

Sat,  t  represents  the  saturation  condition  of  the  image  (natural  or  false); 

Image  m  represents  each  presented  image  and 

Error  n  represents  for  unaccounted  variations  within  the  model. 

C.  RESULTS 

To  determine  the  appropriate  statistical  method  for  the  results  of  the  experiment, 
normality  tests  were  conducted  and  showed  that  the  data  failed  to  follow  the  properties  of 
normality.  Therefore,  the  assumption  that  the  original  data  was  normal  was  not  tenable. 
As  seen  below,  the  analysis  will  proceed  on  intra-subject  differences  in  reaction  times 
and  error  rates  and  while  these  might  not  be  exactly  normal,  the  t-test  is  well-known  to  be 
robust  against  minor  deviations  from  normality. 

Mean  reaction  times  (in  milliseconds)  and  mean  error  rates  (in  percent)  for  each 
of  the  participants  (Appendix  C)  were  calculated  from  individual  performances  for  each 
condition.  To  ensure  participants  did  not  show  any  bias  toward  any  particular  keypress, 
the  mean  error  rates  were  observed  for  the  upright  and  inverted  conditions.  If  the  error 
rates  for  the  upright  representations  were  zero  percent,  it  would  indicate  that  the 
participant  displayed  a  bias  towards  the  "1”  keypress.  If  this  were  the  case,  the  mean 
error  rates  for  the  inverted  orientations  would  be  very  large.  For  this  experiment,  no  bias 
toward  a  particular  keypress  was  noted.  Additionally,  comparing  the  mean  reaction  time 
and  error  rates  for  each  condition  may  show  the  existence  of  a  speed/accuracy  trade-off. 
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A  low  error  rate  and  unusually  large  reaction  time  indicates  that  a  participant  is  very 
careful  in  selecting  the  proper  orientation.  Conversely,  a  high  error  rate  and  small 
reaction  time  indicates  that  a  participant  is  hasty  in  making  decisions.  The  experimental 
data  correlates  with  respective  categorical  speed/accuracy  trade-off  relationships  in  that 
higher  reaction  times  were  generally  accompanied  by  higher  error  rates.  When  images 
were  manipulated  from  their  natural  color  or  gray  conditions,  participant  reaction  time 
and  error  rates  increased.  Box  plots  and  line  plots  of  each  condition  are  seen  in  Figures 
16  and  17.  The  box  plot  was  chosen  because  it  gives  readers  quick  visual  information 
about  the  data  spread  within  a  variable  and  offers  a  comparison  between  variables.  The 
box  plot  consists  of  a  few  basic  elements:  the  upper  (shaded  area)  quartile,  the  lower 
(shaded  area)  quartile  and  the  median  (the  white  line  in  the  middle).  The  whiskers  show 
the  interval  of  values  outside  the  box,  or  values  that  are  no  more  than  one  and  a  half  times 
the  interquartile  range.  The  interquartile  range  is  defined  as  the  difference  between  the 
upper  and  lower  quartile.  The  horizontal  lines  far  outside  the  boxplot  represent  outliers 
(Hamilton,  1992).  The  tabulated  data  can  be  seen  in  Table  1. 


Box  Plot  of  Mean  Reaction  Times  for  all  Conditions 


C  onditions 

Figure  16:  Box  Plot  of  all  image  condition  mean  reaction  times 
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By  inspection  of  the  box  plot  (Figure  16),  the  data  for  the  natural  color  condition 
shows  a  lower  median  reaction  time  and  smaller  variance  than  the  other  image 
conditions.  Outliers  are  shown  to  exist  for  each  condition.  These  outliers  are  attributed 
to  two  specific  participant's  performance,  implying  that  these  participants  were 
consistently  slow  across  each  condition  in  relation  to  other  participants. 


Figure  17:  Line  Plots  of  mean  reaction  times  and  error  rates  for  Color  and  Gray 

conditions. 


The  line  plot  (Figure  17)  offers  a  pictorial  representation  of  Table  1.  Generally, 
the  mean  error  rates  and  reaction  times  are  less  for  color  conditions  than  for  achromatic 
conditions.  The  reaction  times  for  the  color  false  hue  and  gray  false  hue  conditions  do 
not  follow  this  trend,  thereby  displaying  a  possible  effect  for  the  color  false  hue 
condition.  Also,  the  error  rates  between  the  color  false  hue  and  saturation  and  gray  false 
hue  and  saturation  condition  appear  to  be  fairly  consistent  between  conditions,  implying 
that  a  speed/accuracy  trade-off  does  not  account  for  these  results.  The  remaining  error 
rates  and  corresponding  reaction  time  increased  for  images  that  were  viewed  in 
achromatic  formats. 

Table  1  shows  that  the  average  reaction  times  for  the  natural  color  and  natural 
gray  condition  are  smaller  than  the  other  image  conditions.  Smaller  reaction  times  for 
these  conditions  were  expected  by  the  developed  hypothesis  because  true  scene 
representation  is  easier  to  detect  than  scenes  that  do  not  symbolize  the  true  representation 
of  the  scene.  Additionally,  the  error  rate  for  natural  color  imagery  was  smaller, 
indicating  that  participants  who  view  images  under  natural  color  conditions  will  not  only 
respond  faster  but  will  be  more  accurate  in  object  recognition.  Images  that  were 
manipulated  for  hue  and  saturation  produce  longer  reaction  times  and  error  rates,  thereby 
supporting  the  hypothesis  that  manipulating  the  color  of  an  image  will  disrupt  the 
participant's  ability  to  efficiently  process  image  features. 


DIFF 

NC 

COLOR 

CFH 

CFS 

CFHS 

NG 

GRAY 

GFH 

GFS 

GFHS 

Mean  RT 

601.05 

740.03 

664.76 

726.20 

656.52 

673.75 

680.50 

745.52 

SEM 

34.05 

63.13 

51.77 

62.55 

38.13 

53.86 

60.20 

74.94 

Mean  Err 

3.67 

5.33 

7.00 

8.67 

6.33 

8.67 

6.67 

13.00 

SEM 

1.02 

1.33 

1.78 

1.61 

1.64 

1.82 

1.53 

2.49 

Table  1:  Tabulated  data  for  each  conditions  mean  reaction  time  and  error  rate. 
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where  RT  represents  the  reaction  time  of  the  participant  selecting  an  orientation; 

SEM  represents  the  standard  error  of  the  means; 

NC  represents  the  natural  color  condition; 

CFH  represents  the  color  false  hue  condition; 

CFS  represents  the  color  false  saturation  condition; 

CFHS  represents  the  color  false  hue  and  saturation  condition; 

NG  represents  the  natural  gray  condition; 

GFH  represents  the  gray  false  hue  condition; 

GFS  represents  the  gray  false  saturation  condition  and 

GFSH  represents  the  gray  false  hue  and  saturation  condition. 

Because  we  hypothesize  that  there  is  a  difference  in  reaction  time  between  natural 
and  artificial  color  scenes,  the  difference  between  each  condition's  mean  and  natural 
color's  mean  should  be  positive.  Table  2  shows  the  difference  of  the  means  between  each 
condition. 


DIFF 

CFH-NC 

Color 

CFS-NC 

CHFS-NC 

GFH-NG 

Gray 

GFS-NG 

GFHS-NG 

Mean  RT 

138.98 

63.71 

125.14 

17.24 

23.99 

89.00 

SEM 

34.80 

34.80 

35.60 

38.40 

47.59 

53.08 

Mean  Err 

1.67 

3.33 

5.00 

2.33 

0.33 

6.67 

SEM 

1.59 

1.49 

1.67 

2.07 

2.08 

2.37 

Table  2:  Difference  of  the  reaction  time  and  error  rate  means  across  all 
conditions.  Notice  all  values  are  positive  indicating  that  reaction  time 
and  error  rates  for  natural  color  scenes  are  smaller. 


The  difference  of  the  means  of  each  condition  in  Table  2  is  not  the  same; 
therefore,  we  may  conclude  by  inspection  that  there  can  be  an  effect  on  reaction  times  for 
each  condition.  These  positive  values  indicate  that  the  participant,  when  observing 
natural  conditions  will  exhibit  better  performance.  The  comparison  results  displayed  in 
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Table  2  strengthen  the  assumption  that  individual  perception  is  negatively  disrupted  when 
altering  an  images  natural  condition.  To  test  the  data,  a  paired  t-test  using  Bonferroni's 
inequalities  with  an  a  =  0.05,  six  samples  and  a  confidence  level  of  99.2%  was  used. 
When  comparing  the  color  false  hue  condition  versus  the  natural  color  condition  reaction 
time  (t(19)  =  3.9935  p  <  .008)  and  accuracy  (t(  1 9)  =  1.045  p  >  .008),  an  effect  for 
reaction  time  was  found.  This  indicates  that  a  participant's  perception  is  disrupted  when 
changing  the  image's  hue.  Comparing  the  color  false  hue  and  saturation  versus  the 
natural  color  condition  reaction  time  (t(19)  =  3.5148  p  <  .008)  and  accuracy  (t(  1 9)  = 
3.0009  p  <  .008),  an  effect  was  also  found.  Not  only  were  the  participant  reaction  times 
significant,  but  the  error  rates  were  also  significant.  The  fact  that  the  combined  hue  and 
saturation  condition  was  significant  may  mean  that  an  interaction  between  the  conditions 
might  exist.  A  test  for  the  presence  of  interaction  will  be  conducted  later.  The  remaining 
comparisons  from  Table  2,  color  false  saturation  versus  natural  color  condition  reaction 
time  (t(19)  =  1.8308  p  >  .008)  and  accuracy  (t(19)  =  2.2361  p  >  .008),  gray  false  hue 
versus  natural  gray  condition  reaction  times  (t(  1 9)  =  t  =  0.4489  p  >  .008)  and  accuracy 
(t(19)  =  1.1278  p  >  .008),  gray  false  saturation  versus  natural  gray  condition  reaction 
time  (t(19)  =  0.5041  p  >  .008)  and  accuracy  (t(19)  =  0.1599  p  >  .008),  and  gray  false 
hue  and  saturation  versus  natural  gray  condition  reaction  time  (t(19)  =  1.6768  p  >  .008) 
and  accuracy  (t(  1 9)  =  2.8133  p  >  .008)  revealed  no  effect.  The  absence  of  an  effect  for 
color  false  hue  versus  natural  gray  provides  evidence  that  manipulating  an  image  by 
altering  the  saturation  may  not  effect  a  participant's  reaction  time.  This  also  applies  to 
images  viewed  under  monochrome  conditions  where  manipulation  for  saturation  did  not 
affect  a  participant's  perception.  These  results  may  add  evidence  that  when  altering  an 
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image  by  hue  and  saturation,  the  hue  effect  may  influence  the  combined  hue  and 
saturation  effect  on  participant  reaction  time.  Since  achromatic  images  contain  neither 
hue  nor  saturation  levels,  the  statistical  t-tests  agree  with  the  inference  that  there  is  no 
significant  difference  between  the  natural  gray  and  false  gray  conditions. 

Luminance  tests  were  also  conducted  by  comparing  the  false  gray  conditions  to 
the  natural  gray  images.  For  luminance  to  be  a  factor,  the  difference  of  the  means 
between  false  gray  and  natural  gray  conditions  shall  be  a  value  other  than  zero.  Table  2 
shows  that  these  differences  are  positive  and  the  values  are  much  less  than  the  values  of 
the  color  conditions.  However,  as  noted  above,  respective  statistical  t-tests  showed  no 
effect  across  each  false  gray  condition  when  compared  to  the  natural  gray  condition. 
Therefore,  the  effects  of  the  color  conditions  were  due  to  color  and  not  to  a  luminance 
confound. 

To  determine  if  altering  an  image's  hue  had  any  effect  on  participant  reaction  time 
and  error  rates,  the  hue  effect  was  removed  from  the  images  by  subtracting  all  conditions 
of  natural  hue  from  the  corresponding  conditions  of  false  hue.  For  example,  the 
difference  in  the  mean  reaction  time  and  error  rates  for  the  conditions  of  color  false 
saturation  and  both  color  false  hue  and  saturation  removes  the  saturation  effect  and  leaves 
only  the  effect  of  hue  on  the  image.  Under  the  hypothesis,  there  will  be  a  difference  in 
the  reaction  times  of  these  conditions.  Table  3  displays  the  results. 
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DIFF 

CFH-NC 

HUE 

CFHS-CFS  GFH-NG 

GFHS-FGS 

Mean  RT 

138.98 

61.43 

17.24 

65.01 

SEM 

34.80 

43.23 

38.40 

42.03 

Mean  Err 

1.67 

5.00 

2.33 

6.67 

SEM 

1.59 

1.67 

2.07 

2.37 

Table  3:  Difference  in  reaction  time  and  error  rate  means  across 


all  image  conditions  with  and  without  Hue. 

To  test  the  data,  a  paired  t-test  using  Bonferroni's  inequalities  with  an  «=  0.05, 
four  samples  and  a  confidence  level  of  98.75%  was  used.  In  tests  of  the  color  false  hue 
condition  versus  the  natural  color  condition  reaction  time  (t(  1 9)  =  3.9935  p  <  .0125)  and 
accuracy  (t(19)  =  1.045,  p  >.0125),  a  reaction  time  effect  was  found.  This  effect  has 
already  been  explained  above.  The  remaining  comparisons  from  Table  3,  color  false  hue 
and  saturation  versus  color  false  saturation  condition  reaction  time  (t(19)  =  1.4212  p 
>.0125)  and  accuracy  (t(19)  =  0.7373  p  >.0125),  gray  false  hue  versus  natural  gray 
condition  reaction  time  (t(19)  =  0.4489  p>  .0125)  and  accuracy  (t(19)  =  1.1278  p> 
.0125),  and  gray  false  hue  and  saturation  versus  gray  false  saturation  condition  reaction 
time  (t(19)  =  1.5467  p>  .0125)  and  accuracy  (t(19)  =  2.2637p>  .0125)  showed  no 
effect. 


To  determine  if  altering  an  image's  saturation  had  an  effect  on  participant  reaction 
time  and  error  rates,  the  saturation  effect  was  removed  from  the  images  by  subtracting  all 
conditions  of  natural  saturation  from  the  corresponding  conditions  of  false  saturation. 
For  example,  the  difference  in  the  mean  reaction  time  and  error  rates  for  the  conditions  of 
color  false  hue  and  both  color  false  hue  and  saturation  removes  the  hue  effect  and  leaves 
only  the  effect  of  saturation  on  the  image.  Under  the  hypothesis,  there  will  be  a  difference 
in  the  reaction  times  of  these  conditions.  Table  4  displays  the  results. 
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DIFF 

CFS-NC 

SATURATION 
CFHS-CFH  GFS-NG 

GFHS-FGH 

Mean  RT 

63.71 

-13.83 

23.99 

71.76 

SEM 

34.80 

24.93 

47.59 

32.25 

Mean  Err 

3.33 

3.33 

0.33 

4.33 

SEM 

1.49 

1.64 

2.08 

2.12 

Table  4:  Difference  in  reaction  time  and  error  rate  means  across  all 


image  conditions  with  and  without  Saturation. 

To  test  the  data,  a  paired  t-test  using  Bonferroni's  inequalities  with  an  a=  0.05, 
four  samples  and  a  confidence  level  of  98.75%  was  used.  In  tests  of  the  color  false 
saturation  condition  versus  the  natural  color  condition  reaction  time  (t(19)  =  1.8308  p  > 
.0125)  and  accuracy  (t(19)  =  2.2361  p  >  .0125),  color  false  hue  and  saturation  versus 
color  false  hue  condition  reaction  time  (t(19)  =  -.5549  p  >  .0125)  and  accuracy  (t(19)  = 
2.0329  p  >  .0125),  gray  false  saturation  versus  natural  gray  condition  reaction  time  (t(19) 
=  .5041  p  >  .0125)  and  accuracy  (t(  1 9)  =  0.1599  p  >  .0125),  and  gray  false  hue  and 
saturation  versus  gray  false  hue  condition  (t(  1 9)  =  2.2254  p  >  .0125)  and  accuracy  (t(19) 
=  2.0409  p  >  .0125),  failed  to  produce  evidence  that  altering  an  image's  saturation  effects 
participant  reaction  and  error  rates.  Each  condition  indicates  that  whether  viewing  a 
natural  color  image  or  achromatic  image,  an  effect  on  participant  reaction  time  and  error 
rates  when  viewing  an  image  that  has  been  altered  for  saturation  does  not  exist.  This  is 
because  changing  the  image  saturation  does  not  drastically  change  the  image  color,  it 
only  brightens  the  entire  color  spectrum  of  the  image.  It  is  interesting  to  note  that 
participant  reaction  time  when  viewing  a  false  color  image  that  had  been  manipulated  for 
hue  and  saturation  was  less  than  the  reaction  time  when  viewing  a  color  image  that  had 
been  manipulated  for  hue.  This  may  suggest  that  the  brightness  of  the  image  due  to  the 
altered  saturation  level  may  enhance  participant  performance  when  compared  to  a  color 
image  that  was  manipulated  for  hue. 
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To  determine  the  effects  of  color,  we  compare  the  differences  between  the  mean 
reaction  time  and  error  rates  of  each  color  condition  with  the  means  of  the  same  images 
in  the  achromatic  condition.  For  example,  the  difference  in  the  reaction  time  and  error 
rate  means  between  natural  gray  and  the  natural  color  condition  will  be  negligible.  For  a 
condition  of  color  to  be  a  factor,  there  will  be  a  difference  in  the  means  across  each 
condition.  Table  5  displays  the  results. 


NG-NC 

GRAY-COLOR 
GFH-CFH  GFS-CFS 

GFHS-CFHS 

Mean  RT 

55.47 

-66.28 

15.74 

19.32 

SEM 

31.65 

23.74 

45.44 

37.99 

Mean  Err 

2.67 

3.33 

-0.33 

4.33 

SEM 

1.63 

1.71 

2.13 

2.38 

Table  5:  Difference  In  reaction  time  and  error  rate  means  across 


all  image  conditions  between  Color  and  Monochrome. 

To  test  the  data,  a  paired  t-test  using  Bonferroni's  inequalities  with  an  a  =  0.05, 
four  samples  and  a  confidence  level  of  98.75%  was  used.  When  comparing  gray  false 
hue  versus  color  false  hue  condition  reaction  time  (t(  1 9)  =  -2.7912  p  <  .0125)  and 
accuracy  (t(19)  =  1.633  p  >  .0125),  a  reaction  time  effect  is  found.  As  seen  in  Table  5, 
the  reaction  time  for  the  gray  false  hue  image  is  much  smaller  than  the  reaction  time  for 
the  color  false  hue  image.  This  indicates  that  participants  are  more  apt  to  perceive  an 
achromatic  image  faster  than  a  color  image  that  is  altered  for  hue.  This  provides  more 
evidence  that  the  effect  of  hue  is  significant  for  a  colored  image.  The  other  conditions, 
natural  gray  versus  natural  color  condition  reaction  time  (t(19)  =  1.7524  p  >  .0125)  and 
accuracy  (t(19)  =  1.949  p  >  .0125),  gray  false  saturation  versus  color  false  saturation 
condition  reaction  time  (t( 1 9)  =  .3465  p  >  .0125)  and  accuracy  (t(19)  =  -0.1563  p  > 
.0125),  and  gray  false  hue,  saturation  versus  color  false  hue,  saturation  condition  reaction 
time  (t(19)  =  .5086  p  >  .0125)  and  accuracy  (t(  1 9)  =  1.8183  p  >  .0125),  show  no  effect. 
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To  employ  a  second  method  to  validate  the  visual  model,  a  statistically  reliable 
one-sided  t-test  was  conducted  on  the  combination  of  similar  image  conditions  using  an 
a-  0.5.  By  grouping  individual  conditions  of  a  particular  group  (Figure  18)  and 
comparing  these  conditions  reaction  time  and  error  rate  means  that  do  not  possess  the 
groups  stated  manipulation,  the  following  result  is  shown  in  Table  6. 

COMPARING  HUES 


Color  False  Hue 


Color  False  Hue 
and  Saturation 
Gray  False  Hue 


Gray  False  Hue 
and  Saturation 


< 

<C 

< 


> 

> 

> 

> 


Natural  Color 


Color  False 
Saturation 
Natural  Gray 


Gray  False 
Saturation 


Figure  18:  Stacked  comparison  of  all  viewing  conditions 
with  and  without  Hue  manipulated 


Difference  between  Natural  Hue 

and  false  Hue  conditions 

Mean  RT 

72.54 

SEM 

18.69 

Mean  Err 

3.00 

SEM 

1.11 

Table  6:  Reaction  time  and  error 
rate  means  for  images 
manipulated  for  Hue 


Based  on  the  data  provided  in  Table  6,  there  is  evidence  that  a  reaction  time  (t(79) 
=  3.8807  p  <  .05)  and  accuracy  (t(79)  =  2.6982  p  >  .05)  effect  exists.  This  reaction  time 
effect  is  also  present  in  the  previous  tests  that  compare  hue  and  images  that  are  not 
manipulated  for  hue.  These  results  provide  evidence  that  manipulating  an  image's  hue 
will  disrupt  participant  visual  perception. 

To  determine  if  a  saturation  effect  exists,  reaction  time  and  error  rate  means  of  the 
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images  that  contain  natural  saturation  were  compared  to  the  reaction  time  and  error  rate 
means  of  the  images  that  contained  false  saturation.  Figure  19  shows  these  conditions. 

COMPARING  SATURATION 


Color  False 
Saturation 
Color  False  Hue 
and  Saturation 
Gray  False 
Saturation 
Gray  False  Hue 
and  Saturation 
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Natural  Color 


Color  False  Hue 


Natural  Gray 


Gray  False  Hue 


Figure  19:  Stacked  comparison  of  all  viewing  conditions 
with  and  without  Saturation  manipulated 


Difference  between  Natural 
Saturation  and  false  non- 
Saturation  conditions 


Mean  RT 

36.41 

SEM 

17.99 

Mean  Err 

2.83 

SEM 

0.92 

Table  7:  Reaction  time  and  error 
rate  means  for  images 
manipulated  for  Saturation 


The  difference  of  the  mean  reaction  times  between  saturation  conditions  reaction 
time  (t(79)  =  2.0236  p  <  .05)  and  accuracy  (t(79)  =  3.0643  p  <  .05)  shows  an  effect. 
This  result  contradicts  earlier  results  that  test  the  effect  of  saturation  on  participant  visual 


perception.  The  reason  for  this  contradiction  may  be  due  to  the  large  standard  error  for 
the  mean  reaction  time  and  the  increased  degrees  of  freedom  that  occur  by  combining 
image  conditions.  However,  it  must  be  noted  that  the  p-value  (p  =  0.0464)  suggests  that 
the  effect  of  saturation  is  not  very  strong.  There  may  also  be  some  interaction  between 
hue  and  saturation  that  may  enhance  the  effect  of  saturation.  A  statistical  test  for 


interaction  will  be  performed  later. 
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Lastly,  a  comparison  of  color  to  monochrome  conditions  was  conducted  to 
determine  if  a  color  effect  exists.  Although  altering  images  by  changing  color  (hue  and 
saturation)  was  included  with  natural  color  conditions,  this  comparison  will  determine 
whether  color  is  more  preferred  than  achromatic  conditions.  Figure  20  shows  these 
conditions. 


COMPARING  COLOR  TO  GRAY 


Natural  Color 


Color  False  Hue 


Color  False 
Saturation 


Color  False  Hue 
and  Saturation 
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Natural  Gray 


Gray  False  Hue 


Gray  False 
Saturation 


Gray  False  Hue 
and  Saturation 


Figure  20:  Stacked  comparison  of  all  viewing  conditions 
with  and  without  Color 


Difference  between  Color  and 

achromatic  conditions 

Mean  RT 

-6.06 

SEM 

18.17 

Mean  Err 

2.50 

SEM 

0.99 

Table  8:  Reaction  time  and  error 
rate  means  for  images  with  and 
without  Color 


Observing  the  Table  8  data  for  comparing  color  conditions  with  achromatic 
conditions  reaction  time  (t(79)  =  -.3337  p  >  .05)  and  accuracy  (t(79)  =  2.5131  p  <  .05) 
an  effect  was  found.  The  negative  value  for  the  mean  reaction  times  indicates  that 
participant  reaction  times  are  slightly  smaller  when  viewing  images  in  achromatic  format 
when  compared  to  a  color  format. 

Strong  evidence  has  already  been  shown  that  supports  the  fact  that  altering  an 
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image's  hue  has  a  pronounced  effect  on  participant’s  perceptual  abilities.  However, 
previous  results  indicate  that  an  interaction  may  exist  between  hue  and  saturation  when 
images  are  altered  by  these  combined  conditions.  Using  the  hypothesis  that  there  exists 
interaction  between  conditions  of  hue  and  saturation,  the  following  linear  model  is 
developed: 

Reaction  Time  ijkImn  =  Part .  +  Color  j  +  (Hue  *  *  Sat , )  +  Image  m  +  error  „ 
where  Part .  represents  the  ith  participant  of  the  experiment. 

Color  .  represents  the  color  of  the  image  (natural  or  monochrome); 

(Hue*  *Sat; )  represents  the  interaction  between  the  two  variables; 

Image  m  represents  each  presented  image  and 

Error  „  represents  for  unaccounted  variations  within  the  model. 

By  combining  the  effects  of  hue  and  saturation  and  running  the  linear  model 
to  test  for  interaction  between  the  variables,  no  interaction  is  shown  to  exist  (t(136)  — 
-0.3968  p  >  .05).  The  individual  values  for  hue  (t(136)  =  -2.3837  p  <  .05)  and 
saturation  (t(136)  =  -1.0921  2  ^  -05)  also  confirm  the  effect  that  altering  hue  will 
disrupt  participant  reaction  time  and  error  rates.  This  interaction  model  is  strongly 
supported  by  a  high  coefficient  of  determination  (R2 )  of  81  percent,  which  shows  the 
amount  of  the  total  variance  explained  by  the  model.  Therefore,  it  is  concluded  that 
the  effects  for  hue  have  a  significant  effect  on  participant  reaction  time.  When  an 
image  is  manipulated  for  both  hue  and  saturation,  they  act  independently  of  each 
other  and  it  is  the  effect  of  hue  that  causes  the  increase  in  participant  reaction  time 


and  error  rates  to  increase. 


D.  DISCUSSION 


The  purpose  of  this  experiment  is  to  clarify  the  role  of  color  in  object  recognition. 
Previous  research  has  been  inconclusive  in  adequately  defining  colors  role.  However, 
equipped  with  a  large  abundance  of  professionally  photographed  images,  this  experiment 
makes  it  possible  to  manipulate  the  images  in  eight  different  conditions  for  comparison. 
It  was  hypothesized  that  natural  color  scenes  would  be  interpreted  in  a  similar  manner  to 
scenes  displayed  achromatically.  However,  an  artificial  color  scene  altered  for  hue, 
saturation,  or  both  hue  and  saturation,  would  degrade  a  participant’s  visual  perception 
process  when  compared  to  either  natural  color  or  natural  gray. 

The  methodology  for  this  experiment  originates  by  attempting  to  clarify  colors 
role  in  the  "real-world."  Previous  research  defined  colors  role  in  controlled  laboratory 
environments  without  using  natural  scenes,  and  defined  color  as  a  combination  of  natural 
color  and  artificial  color.  Natural  objects  were  defined  as  living,  and  unnatural  objects 
defined  as  non-living.  Object  recognition  failed  to  emphasize  the  role  of  color  in  global 
scenes. 

This  experiment  varied  the  chromatic  format  of  the  entire  scene.  Not  only  were 
individual  objects  altered  in  color,  but  the  background  and  foregrounds  were  altered  as 
well.  Images  were  defined  as  natural  if  they  were  unaltered  and  artificial  if  they  were 
altered  in  hue,  saturation,  or  both.  To  provide  a  control  measure,  achromatic 
representations  were  developed  for  each  color  format.  Additionally,  luminance  effects 
were  determined  by  the  comparison  of  natural  gray  to  the  other  manipulated  gray 
conditions.  The  experimental  design  employed  a  forced  choice  task,  where  observers 
were  to  indicate  scene  orientation. 

Perhaps  the  preferred  method  in  discussing  the  results  of  Experiment  1  is  to 


65 


analyze  the  data  from  the  combined  or  stacked  image  comparisons,  then  to  analyze  the 
data  from  individual  image  comparisons.  This  type  of  discussion  will  first  show  that 
generalizing  color  as  both  natural  and  artificial  will  display  results  that  are  different  when 
color  is  analyzed  in  the  respective  individual  natural  and  artificial  format. 

The  results  of  analyzing  the  combined  comparisons  between  color  and  achromatic 
scenes  produced  an  effect  for  the  error  rate  means.  Although  the  reaction  times  means 
were  slightly  less  for  achromatic  images,  participants  error  more  frequently  which 
significantly  affected  their  performance.  This  result  suggests  that  color  is  beneficial  in 
object  recognition  and  validates  the  majority  of  previous  research. 

It  is  wrong  to  suggest  that  these  findings  be  applied  to  studies  in  which  artificial 
color  is  tested.  The  fact  that  previous  research  suggests  that  the  HVS  is  disrupted  when 
viewing  non-prototypical  objects,  applying  generalized  color  findings  to  experiments  that 
alter  color,  is  misleading.  The  HVS  develops  color  mappings  that  may  be  recalled  when 
necessary.  When  a  conflict  occurs  between  the  recognition  of  a  particular  object  and  its 
stored  color  mapping  features,  a  disruption  occurs  in  the  participant's  ability  to  efficiently 
recognize  the  objects.  The  results  of  the  comparisons  for  images  that  are  altered  for  hue 
and  those  that  are  not  manipulated  for  hue  are  significant.  This  finding  suggests  that  once 
a  natural  scene  is  altered  by  changing  the  color,  the  HVS  is  disrupted.  Analysis  of  the 
images  that  were  altered  for  saturation  was  also  significant.  This  result  is  surprising 
because  when  the  saturation  of  the  natural  scene  is  modified,  the  brightness  and  not 
necessarily  the  color,  is  changed.  However,  it  is  shown  that  when  an  image  is  altered  for 
hue  and  saturation,  the  effect  for  hue  overwhelms  and  effect  for  saturation  and  causes  the 
combined  hue  and  saturation  conditions  to  be  significant,  implying  that  either  or  both  the 
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hue  and  saturation  are  disruptive.  The  results  of  the  statistical  tests  for  saturation  are  not 
strong  and  therefore,  cannot  suggest  that  image  manipulation  for  saturation  significantly 
disrupts  the  HVS.  More  studies  should  be  conducted  to  validate  this  finding. 

Another  statistical  method  tested  the  comparisons  of  each  image  format  against 
combinations  of  the  other  formats  to  find  an  effect  for  hue,  saturation,  and  color.  Each 
manipulated  condition  was  compared  to  the  natural  format,  whether  it  was  color  or 
monochrome.  The  differences  in  the  mean  reaction  times  and  error  rates  showed  a 
significant  effect  for  color  false  hue  and  color  false  hue  and  saturation.  The  non¬ 
significant  results  of  the  achromatic  comparison  shows  that  luminance  has  no  effect  on 
participant  perceptual  ability.  Because  the  effect  for  both  hue  and  saturation  is 
significant,  the  possibility  of  interaction  between  hue  and  saturation  must  be  considered. 
A  linear  model  was  developed  and  results  failed  to  show  any  hue  and  saturation 
interaction.  Therefore,  more  evidence  exists  that  suggests  altering  an  image  for  hue 
disrupts  the  participant’s  ability  to  recognize  objects.  Image  manipulation  for  saturation 
was  then  tested.  The  statistical  results  show  a  lack  of  evidence  to  suggest  any  effect  on 
participant  performance.  Image  manipulation  for  hue  was  then  tested.  Effects  for  hue 
were  already  found  in  the  combined  or  stacked  tests  as  well  as  the  comparisons  to  the 
natural  formats.  The  results  of  hue  manipulation  shows  that  an  image  altered  for  hue  has 
an  effect  when  compared  to  images  in  the  natural  format.  Again,  this  suggests  that 
altering  the  natural  color  of  a  scene  adversely  effects  the  participants  ability  to  process 
the  scene  with  the  same  efficiency  as  in  natural  color.  Lastly,  the  specific  color  format 
manipulations  were  compared  to  the  achromatic  conditions.  The  results  showed  that 
changing  the  hue  affects  the  HVS  when  it  is  compared  to  the  same  scene  in  achromatic 
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format.  One  interesting  comparison  to  note  is  that  no  effect  is  found  between  a  natural 
color  scene  and  an  achromatic  scene.  Although  only  15  images  were  compared  in  these 
formats,  this  result  adds  validity  to  the  hypothesis  that  natural  color  does  not  enhance 
object  recognition  when  compared  to  a  natural  monochrome  scene. 

The  results  of  Experiment  1  showed  that  depending  on  the  specific  experimental 
design  employed,  different  results  will  occur.  Grouping  images  by  color  and  comparing 
them  to  images  in  achromatic  format,  performance  with  color  imagery  was  found  to  be 
better.  This  result  validates  numerous  color  theories  concerning  object  recognition. 
However,  when  separating  the  colors  into  natural  and  artificial  categories,  evidence 
shows  that  color  may  not  be  beneficial.  As  stated,  the  type  of  experimental  design 
employed  reveals  different  results.  Due  to  the  overwhelming  evidence  that  implies  that 
altering  an  images  natural  color  will  adversely  affect  participant  performance  in  object 
recognition,  the  image  format  that  is  most  preferred  is  the  natural  format.  If  scenes 
cannot  be  represented  in  the  natural  color,  then  the  best  alternative  is  to  display  the  scene 
achromatically. 
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IV.  EXPERIMENT  2 


A.  METHODS 

1.  Participants 

As  in  Experiment  1,  fifteen  students  voluntarily  participated  in  this  study.  The 
participants  did  not  take  part  in  the  previous  experiment. 

2.  Apparatus 

The  same  experimental  workstation  used  in  the  previous  experiment  was  used  in 
Experiment  2. 

3.  Stimuli 

Each  of  270  images  (30  of  which  were  designated  for  practice)  was  presented  in 
one  of  eight  different  formats.  These  formats  were  natural  color  and  gray,  and  color  and 
false  gray  with  hue.  Additionally,  half  of  the  images  in  each  cell  were  presented  with 
Gaussian  noise  added.  Images  were  also  presented  in  an  upright  or  inverted  position,  so 
that  each  image  was  presented  in  16  possible  ways. 

The  experiment  was  constructed  by  examining  the  previous  experiment  stimuli 
and  choosing  the  images  that  contained  salient  color  contrast  as  judged  by  the  author  and 
one  other  vision  science  expert.  Objects  within  the  images  that  encompassed 
approximately  25%  or  more  of  the  total  viewing  area  were  eliminated.  These  two 
selection  processes  eliminated  approximately  half  the  previous  experiment  images. 
Replacement  images  were  selected  from  the  same  categories  of  images  off  the  Softkey 
Photo  Gallery  CD.  The  images  were  cropped  to  a  rectangular  250  X  400  pixel  size,  then 
manipulated  using  the  Photoshop  software's  standard  image  processing  functions.  Image 
manipulations  to  generate  the  color  false  hue,  natural  gray  and  gray  false  hue  conditions 
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were  identical  to  those  used  in  Experiment  1.  Noise  was  added  by  increasing  or 
decreasing  the  assigned  value  of  each  pixel  by  an  amount  chosen  randomly  from  a 
Gaussian  distribution  with  mean  zero  and  standard  deviation  of  50.  Pixel  values  of  more 
than  255  or  less  than  zero  were  set  to  values  of  255  or  zero  respectively. 


Procedures  to  convert  the  images  to  achromatic  format  were  identical  to  those  of 
the  previous  experiment.  Figure  21  depicts  each  image  representation. 


Figure  21:  Image  comparison  with  and  without  noise.  A)  Natural  Color  with  Noise, 
B)  Color  False  Hue  with  Noise,  C)  Natural  Gray  with  Noise,  D)  Gray  False  Hue  with 


Noise 

4.  Procedure 

The  same  procedure  used  for  the  previous  experiment  was  used  in  Experiment  2. 
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5.  Experimental  Design 

The  design  for  Experiment  2  was  a2X2X2X2  within-subjects  factorial  design. 
Factors  included  orientation  (upright  or  inverted),  color  (color  or  gray),  hue  (natural  or 
false)  and  noise  (noise  or  no  noise).  The  observations,  number  of  cells,  and  data 
collection  methods  were  identical  to  Experimental  1. 

B.  DEVELOPMENT  OF  VISUAL  MODEL 

Inconsistencies  will  also  occur  between  each  subject's  response  time  and  error 
rates  similar  to  the  data  from  the  previous  experiment.  To  account  for  the  differences,  the 
following  model  is  proposed: 

Reaction  Time, jjklmn  =  Part,  +  Color  .  +  Hue k  +  Noise ,  +  Image m  +  error n 
where  Part,  represents  the  ith  participant  of  the  experiment; 

Color  .  represents  the  color  of  the  image  (natural  or  monochrome); 

Hue  k  represents  the  hue  condition  of  the  image  (natural  or  false); 

Noise,  represents  the  noise  condition  of  the  image  (noise  or  no  noise); 

Image  m  represents  each  presented  image  and 

Error  n  represents  for  unaccounted  variations  within  the  model. 

C.  RESULTS 

To  remain  consistent  with  statistical  testing  employed  in  Experiment  1,  the  t-test 
with  level  of  significance  of  a  =  .05  was  used.  The  reaction  times  (in  milliseconds)  and 
error  rates  (in  percent)  for  each  of  the  participants  (Appendix  D)  were  calculated  to 
obtain  the  reaction  time  and  error  rate  means  for  each  condition  of  the  experiment.  As  in 
Experiment  1,  reaction  time  and  error  rate  means  were  compared  to  determine  participant 
speed/accuracy  trade-off  effects.  Additionally,  since  error  rates  were  not  constant  for 
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Reaction  Times 


each  inverted  condition,  bias  effects  were  determined  to  be  non-existent.  A  box  plot  and 
line  plot  of  each  condition  is  shown  in  Figures  22  and  23  to  provide  the  reader  a  rapid 
view  of  the  data. 


Box  Plot  of  Mean  Reaction  Times  for  all  Conditions 


O 

§1 


NC  NCNOISE  CFH  CFH  Noise  NG  NGNOISE  GFH  GFHNoise 


C  onditions 

Figure  22:  Box  Plot  of  all  image  conditions  mean  reaction  times 
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Color  and  Gray  Conditions  Mean  Error  Rates 


Figure  23:  Line  Plots  of  mean  reaction  times  and  error  rates  for  Color  and  Gray 
conditions  with  "No  Noise"  and  "Noise." 


DIFF 

NATURAL 
NC  NCNOISE 

CFH 

CFHNOISE 

NG 

GRAY 

NGNOISE  GFH 

GFHNOISE 

MEAN  RT 

742.65 

817.78 

841.66 

1003.24 

803.94 

855.38 

899.29 

917.00 

SEM 

44.72 

59.15 

50.67 

74.33 

42.34 

47.56 

36.38 

76.77 

MEAN  Err 

2.22 

9.78 

8.89 

13.33 

7.56 

20.00 

11.55 

14.22 

SEM 

1.06 

2.59 

1.25 

2.84 

1.70 

2.25 

2.00 

2.14 

Table  9:  Tabulated  data  for  image  condition  reaction  time  and  error  rate  means 

Where  NCNOISE  represents  natural  color  with  noise; 

CFHNOISE  represents  color  false  hue  with  noise; 

NGNOISE  represents  natural  gray  with  noise  and 
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GFHNOISE  represents  gray  false  hue  with  noise. 

By  inspection  of  the  box  plot,  a  large  difference  in  each  condition's  reaction  time 
medians  across  all  conditions  is  seen.  This  suggests  that  participant  response  rates  are 
affected  by  a  particular  image  presentation.  The  large  variance  of  the  data  also  suggests 
that  participant  performance  is  affected.  The  interquartile  range  comparison  between 
conditions  of  noise  and  those  conditions  without  noise  shows  that  higher  uncertainty  in 
response  times  exists  for  noisy  conditions.  It  is  interesting  to  note  that  the  false  color  and 
false  gray  noise  conditions  that  were  manipulated  for  hue  are  excessively  large  which 
suggests  a  severe  disruption  of  participant  performance  when  viewing  these  particular 
image  conditions.  There  are  fewer  outliers  displayed  than  what  was  seen  in  the  previous 
experiment,  presumably  because  the  data  is  more  dispersed  overall.  A  single  participant 
was  responsible  for  every  outlier  except  for  one.  The  mean  error  rates  observed  in  Table 
9  shows  a  relationship  that  corresponds  to  the  mean  reaction  times.  The  speed/accuracy 
relationship  between  conditions  with  noise  and  conditions  without  noise  shows  that 
participant's  reaction  time  and  error  rates  increased  with  the  addition  of  noise.  This 
implies  that  there  was  no  speed/accuracy  trade-off.  For  example,  given  that  there  is 
approximately  a  52-millisecond  difference  in  reaction  time  between  conditions  of  natural 
gray  with  and  without  noise,  the  error  rate  for  the  noisier  condition  is  approximately  three 
times  that  of  the  no  noise  condition.  Therefore,  it  is  concluded  that  under  conditions 
where  error  rates  are  similar,  the  difference  in  reaction  times  between  conditions  of 
natural  gray  with  no  noise  and  natural  gray  conditions  with  noise  would  be  much  greater. 
The  line  plots  seem  to  confirm  this  observation.  This  effect  applies  to  the  hue  and  color 
conditions  with  noise  when  compared  to  the  same  conditions  without  noise,  although  not 
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at  the  same  magnitude. 

Before  statistical  analysis  was  conducted  to  determining  whether  noise  has  an 
effect  on  reaction  times  across  all  image  conditions,  the  data  obtained  from  images 
without  noise  was  used  to  validate  portions  of  the  model  used  in  the  previous  experiment. 
The  previous  experiment  showed  that  altering  an  image's  saturation  has  no  effect  on 
participant  reaction  time  but  altering  the  hue  of  an  image  does  effect  performance. 
Comparing  the  difference  in  reaction  time  and  error  rate  means  for  hue  will  determine  if 
using  additional  images  validates  the  findings  in  Experiment  1.  Under  the  hypothesis, 
there  is  a  difference  in  the  reaction  times  of  these  conditions.  Table  10  shows  this 
comparison. 


DIFF 

CFH-NC 

HUE 

GFH-NG 

CFHNOISE- 

NCNOISE 

GFHNOISE- 

NGNOISE 

MEAN  RT 

99.01 

95.36 

185.47 

17.70 

SEM 

25.24 

29.68 

43.65 

56.62 

MEAN  Err 

6.67 

4.00 

3.55 

-5.78 

SEM 

1.45 

2.82 

2.42 

2.67 

Table  10:  Difference  in  reaction  time  and  error  rate  means  across  all  image 

conditions  with  and  without  Hue. 


It  is  seen  that  there  is  a  positive  difference  across  all  image  conditions  that  are 
manipulated  for  hue,  similar  to  the  results  for  the  previous  experiment.  This  adds 
additional  evidence  that  altering  an  image's  hue  causes  participant  reaction  times  to 
increase.  There  is  a  large  difference  between  the  color  false  hue  noise  and  the  natural 
color  noise  condition.  This  indicates  that  not  only  does  changing  hue  have  an  effect  on 
reaction  time,  but  noise  increases  that  effect.  To  test  the  data,  a  paired  t-test  using 
Bonferroni's  inequalities  with  an  a=  0.05,  four  samples  and  a  confidence  level  of 
98.75%  was  used.  When  comparing  the  color  false  hue  versus  the  natural  color  condition 
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reaction  time  (t(14)  =  3.9232  2  ^  .0125)  and  accuracy  (t(14)  =  4.5833  2  <  .0125),  color 
false  hue  with  noise  versus  natural  color  with  noise  condition  reaction  time  (t(14)  = 
4.2491  2  ^  .0125)  and  accuracy  (t(14)  =  1.4176  2  ^  .0125),  gray  false  hue  versus  natural 
gray  condition  reaction  time  (t(14)  =  3.2129  2  ^  .0125)  and  accuracy  (t(14)  =  1.467  2  ^ 
.0125),  an  effect  is  found.  These  results  validate  the  findings  from  Experiment  1.  The 
exception  is  that  in  Experiment  1,  no  effect  was  found  between  gray  false  hue  and  natural 
gray.  For  an  unknown  reason,  there  is  a  larger  reaction  time  and  error  rate  difference  in 


this  experiment  than  compared  to  the  previous  experiment.  No  explanation  can  be  made 
concerning  this  result.  The  gray  false  hue  with  noise  versus  natural  gray  with  noise 
condition  reaction  time  (t(14)  =  1.3281  2  ^  .0125)  and  accuracy  (t(14)  =  -2.1619  2  ^ 
.0125)  shows  no  effect. 

To  validate  the  effect  of  color,  we  compare  the  differences  between  the  means  of 
each  color  condition  with  the  same  images  in  the  achromatic  condition  as  seen  in 
Experiment  1.  For  a  condition  of  color  to  be  a  factor,  there  will  be  a  difference  in  the 
means  across  each  condition.  Table  1 1  displays  the  results. 


GRAY-COLOR 

NGNOISE-  GFHNOISE- 


DIFF 

NG-NC 

NCNOISE 

GFH-CFH 

CFHNOISE 

MEAN  RT 

61.29 

37.60 

57.63 

-86.25 

SEM 

16.05 

38.30 

32.67 

55.00 

MEAN  Err 

5.33 

10.22 

2.67 

0.89 

SEM 

1.62 

3.11 

1.93 

2.24 

Table  11:  Difference  in  reaction  time  and  error  rate  means  across  all  image 
conditions  between  Color  and  Monochrome 


It  is  shown  that  there  exists  a  difference  across  all  conditions.  A  significant 
finding  in  this  experiment  that  differs  from  the  previous  experiment  findings  is  that  the 
reaction  time  difference  between  the  gray  false  hue  and  color  false  hue  is  positive 
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indicating  that  participants  viewed  the  achromatic  format  slower.  The  only  explanation 
for  this  contradiction  may  be  that  participants  reacted  with  more  caution  in  selecting  the 
proper  orientation.  An  error  rate  comparison  between  the  gray  false  hue  and  color  false 
hue  shows  that  the  error  rate  is  less  in  this  experiment.  This  delay  in  image  selection 
contributes  to  the  increase  in  reaction  time  and  reduced  error  rate.  The  remaining 
conditions  contain  a  positive  value  indicating  that  images  viewed  with  color  will  enable  a 
participant  to  respond  faster.  In  tests  of  the  data,  a  paired  t-test  using  Bonferroni's 
inequalities  with  an  a=  0.05,  four  samples  and  a  confidence  level  of  98.75%  was  used. 
When  comparing  the  natural  gray  versus  the  natural  color  condition  reaction  time  (t(14)  = 
3.8183  p  <  .0125)  and  accuracy  (t(14)  =  3.2922  p  <  .0125),  and  natural  gray  with  noise 
versus  natural  color  with  noise  condition  reaction  time  (t(14)  =  0.9818  p  >  .0125)  and 
accuracy  (t(14)  =  3.2859  p  <  .0125)  an  effect  is  shown.  The  natural  gray  with  noise 
versus  natural  color  with  noise  condition  effect  is  contrary  to  the  findings  in  the  previous 
experiment.  An  explanation  for  this  result  may  be  due  to  the  object  size  restriction  when 
selecting  experimental  stimuli.  Without  the  benefit  of  large  objects,  color  played  a  key 
role  in  segmenting  global  features  contained  in  the  degraded  images.  The  remaining 
conditions  gray  false  hue  versus  color  false  hue  condition  reaction  time  (t(14)  =  1.7642  p 
>  .0125)  and  accuracy  (t(14)  =  1.3813  p  >  .0125),  and  gray  false  hue  with  noise  versus 
color  false  hue  with  noise  condition  reaction  time  (t(14)  =  -1.5682  p  >  .0125)  and 
accuracy  (t(14)  =  0.3974  p  >  .0125)  showed  no  effect. 

Finally,  to  compare  an  image  that  is  spatially  degraded  by  noise  to  an  image  that 
does  not  suffer  from  spatial  degradation,  we  will  determine  if  an  effect  of  Gaussian  noise 
exists.  It  is  hypothesized  that  there  is  a  difference  in  participant  reaction  time  and  error 
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rates  between  these  types  of  images.  By  taking  the  difference  of  the  means,  there  will  be 
a  difference  in  reaction  times  between  each  condition  for  noise.  Table  12  displays  the 


results. 


NOISE 

NCNOISE-  CFHNOISE-  NGNOISE-  GFHNOISE- 

DIFF  NC  CFH  NG _ GFH 


MEAN  RT 

75.13 

161.58 

51.44 

17.70 

SEM 

23.41 

42.27 

37.19 

56.62 

MEAN  Err 

7.56 

4.44 

12.44 

2.67 

SEM 

2.24 

2.81 

2.33 

2.97 

Table  12:  Difference  in  reaction  time  and  error  rate  means  across  all  image 
conditions  with  Noise  and  images  without  Noise 

There  exists  an  effect  for  noise  under  all  image  formats.  In  fact,  there  is  a  huge 
variability  in  the  effect  of  noise  on  reaction  times  and  error  rate  means  with  ranges  from 
17.70  to  161.58  milliseconds  and  2.67  to  12.44  percent  respectively.  The  standard  errors 
for  the  reaction  time  means  are  very  high  and  inconsistent  across  each  condition.  This 
shows  that  participants  view  images  without  noise  much  faster  and  more  accurately  than 
images  that  contain  noise.  All  conditions  contain  a  positive  value  that  indicates  that 
images  that  do  not  contain  noise  will  decrease  participant  reaction  time.  To  test  the  data, 
a  paired  t-test  using  Bonferroni's  inequalities  with  an  a-  0.05,  four  samples  and  a 
confidence  level  of  98.75%  was  used  to  test  the  data.  When  comparing  the  natural  color 
with  noise  versus  the  natural  color  condition  reaction  time  (t(14)  =  3.2093  p  <  .0125)  and 
accuracy  (t(14)  =  3.3712  p  <  .0125),  color  false  hue  with  noise  versus  the  color  false  hue 
condition  reaction  time  (t(14)  =  3.8228  p  <  .0125)  and  accuracy  (t(14)  =  1.5811  p  > 
.0125),  and  natural  gray  noise  versus  the  natural  gray  condition  reaction  time  (t(14)  = 
1.3831  p  >  .0125)  and  accuracy  (t(14)  =  5.3324  p  <  .0125),  an  effect  exists.  However, 
when  comparing  the  gray  false  hue  noise  versus  the  gray  false  hue  condition  reaction 
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time  (t(14)  =  0.3127  2  ^  .0125)  and  accuracy  (t(14)  =  0.8993  2  >  .0125)  no  effect  is 
shown.  This  implies  that  color  images  that  lack  visual  clarity  are  effected  by  this 
deficiency  more  than  achromatic  images. 

To  further  validate  the  visual  model  for  this  experiment,  the  second  method  of 


statistical  analysis  is  employed  in  the  previous  experiment  was  repeated.  A  combined 
one-sided  t-test  was  conducted  for  individual  image  conditions  using  an  a-  0.5.  By 
grouping  individual  conditions  of  a  particular  group  (Figure  24)  and  comparing  these 
conditions  reaction  time  and  error  rate  means  to  those  conditions  that  do  not  possess  the 


groups  stated  manipulations,  the  following  results  are  shown  in  Table  13. 
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Figure  24:  Stacked  comparison  of  all  viewing  conditions 
with  and  without  Hue  manipulated 


Difference  between  Hue  and 

non-Hue  conditions 

Mean 

110.36 

SEM 

19.14 

Mean  Err 

2.11 

SEM 

1.32 

Table  13:  Reaction  time  and  error 
rate  means  for  images 
manipulated  for  Hue 


Based  on  the  data  provided  in  Table  13,  there  is  evidence  that  a  reaction  time 
(t(59)  =  5.767  2  ^  -05)  and  accuracy  (t(59)  =  1.5977  2  ^  -05)  effect  exists.  This  effect  is 
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also  present  in  Experiment  1  and  is  validated  in  this  experiment.  These  results  provide 
evidence  that  manipulating  an  image's  hue  will  disrupt  participant  visual  perception. 

To  determine  if  a  color  effect  exists,  reaction  time  and  error  rate  means  for  color 
are  compared  to  the  achromatic  images.  This  comparison  will  determine  whether  color  is 
more  beneficial  in  object  recognition  than  in  achromatic  images.  Figure  25  shows  these 
conditions. 


COMPARING  COLOR 
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Figure  25:  Stacked  comparison  of  all  viewing  conditions 
with  and  without  Color 


Difference  between  Color  and 

achromatic  conditions 

Mean 

-17.57 

SEM 

20.18 

Mean  Err 

4.78 

SEM 

1.21 

Table  14:  Reaction  time  and  error 
rate  means  for  images  with  and 
without  Color 


As  observed  in  Table  14,  there  exists  significant  evidence  of  an  accuracy  effect 
for  the  reaction  time  (t(59)  =  -0.8706  p> .05)  and  accuracy  (t(59)  =  3.9562  p<.05).  In 
fact,  as  seen  in  the  previous  experiment,  participants  respond  to  achromatic  images 
slightly  faster  than  colored  images.  This  result  can  be  misleading  given  that  it  has  been 
shown  that  altering  an  images  hue  affects  participant  reaction  time.  As  described  in  the 
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previous  experiment,  by  increasing  the  degrees  of  freedom  by  combining  all  color 
conditions  and  comparing  reaction  time  and  error  rate  means  against  all  achromatic 
condition  reaction  time  and  error  rate  means,  the  effect  of  hue  is  reduced  by  the  addition 
of  the  other  conditions. 

Lastly,  to  examine  the  effect  of  noise,  reaction  time  and  error  rate  means  are 
compared.  Figure  26  shows  these  conditions. 

COMPARING  NOISE 


Natural  Color 


Color  False  Hue 


Natural  Gray 


Gray  False  Hue 
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Natural  Color 
with  Noise 
Color  False  Hue 
with  Noise 
Natural  Gray  with 

_ Noise _ 

Gray  False  Hue 
with  Noise 


Figure  26:  Stacked  comparison  of  all  viewing  conditions 
with  and  without  Noise 


Difference  between  Images  with 
Noise  to  Images  without  Noise 

Mean 

76.46 

SEM 

21.42 

Mean  Err 

6.78 

SEM 

1.36 

Table  15:  Reaction  time  and  error 
rate  means  for  images  with  and 
without  Noise 


As  observed  in  Table  15,  there  exists  evidence  of  a  significant  effect  for  reaction 
time  (t(59)  =  3.5702  p<.05)  and  accuracy  (t(59)  =  4.9897  p<.05).  It  is  also  seen  that 
images  with  noise  are  responded  to  slower  than  clear  images. 

D.  DISCUSSION 

The  purpose  of  this  experiment  is  to  clarify  the  role  of  color  in  object  recognition 
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when  viewing  spatially  degraded  imagery.  It  is  hypothesized  that  natural  color  would 
enhance  object  recognition  for  images  that  are  degraded  with  noise. 

The  principle  research  that  questioned  the  role  of  color  in  object  recognition  was 
conducted  by  Wurm,  et  al.,  (1993).  This  research  used  degraded  food  images  to 
determine  color's  role.  It  was  found  that  color  aided  the  recognition  of  objects  in  blurred 
and  unblurred  conditions. 

This  experiment  used  natural  scenes  for  stimuli  taken  from  a  professionally 
photographed  commercial  CD.  The  reason  for  using  natural  scenes  was  to  closely 
approximate  imagery  that  may  be  found  on  military  night  vision  displays.  These  displays 
are  small  which  hinders  image  resolution.  The  images  were  presented  in  either  natural 
color  or  achromatic  condition  or  manipulated  by  altering  the  hue.  Gaussian  noise  was 
also  added  to  half  of  the  images. 

Analysis  of  the  data  provided  a  means  to  validate  the  findings  of  Experiment  1. 
When  comparing  the  combined  conditions  of  natural  hue  with  unnatural  hue,  an  effect 
was  found.  This  effect  validates  the  findings  in  the  previous  experiment.  These 
overwhelming  results  strongly  suggest  that  the  HVS  is  disrupted  when  an  images  natural 
color  is  altered.  Individually,  all  image  conditions  of  false  hue  were  compared  to  the 
natural  hue  conditions.  The  findings  of  this  experiment  validate  the  findings  obtained  in 
Experiment  1. 

When  the  combined  conditions  of  color  were  compared  to  the  achromatic 
conditions,  an  effect  was  found.  This  effect  validates  previous  experiment  findings. 
When  comparing  images  that  are  achromatic  with  those  that  display  color,  either  false  or 
natural,  color  appears  to  play  an  important  role  in  object  recognition.  Individually,  all 


82 


image  conditions  of  false  hue  were  compared  to  natural  hue  conditions.  The  findings  of 
this  experiment  validate  the  findings  of  Experiment  1.  When  a  natural  color  scene  is 
modified  by  altering  the  color,  independent  of  image  clarity,  the  HVS  is  negatively 
disrupted  by  the  color  change.  However,  the  findings  in  this  experiment  also  revealed  an 
effect  between  gray  false  hue  versus  natural  gray  that  contradicts  the  findings  in 
Experiment  1.  No  adequate  explanation  can  be  determined  to  explain  this  effect. 

All  conditions  were  individually  compared  for  color  and  the  results  differed  from 
the  color  results  found  in  Experiment  1.  An  effect  was  found  between  natural  color  and 
natural  gray  conditions,  independent  of  image  clarity.  The  most  likely  explanation  for 
the  significance  of  color  in  object  recognition  found  in  this  experiment  is  that  objects 
contained  in  the  natural  scenes  are  smaller  and  less  pronounced  as  compared  to  objects  in 
Experiment  1  stimuli.  A  criterion  for  this  experiment  was  that  objects  could  not 
encompass  more  than  approximately  25  percent  of  the  visual  field  of  view.  This 
reduction  in  object  size  required  the  participant  to  globally  process  the  entire  scene 
instead  of  identifying  a  particular  object.  Therefore,  spatial  segmentation  of  individual 
objects  was  nullified  and  global  scene  processing  occurred.  It  was  shown  that  natural 
color  was  beneficial  in  scenes  that  required  global  processing.  However,  when  the  color 
is  altered,  no  effect  is  evident  when  compared  to  the  achromatic  format.  The  reaction 
times  for  color  false  hue  were  much  less  than  the  reaction  times  for  gray  false  hue  in  this 
experiment.  This  result  is  completely  opposite  to  the  comparison  found  in  Experiment  1. 
Although  not  significant,  it  suggests  that  any  type  of  color  may  benefit  object  recognition 
and  the  HVS  is  disrupted  less  for  scenes  that  are  processed  globally  using  artificial  color. 

A  significant  effect  exists  between  scenes  that  are  spatially  degraded  and  scenes 
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that  contain  adequate  resolution.  This  result  validates  previous  research  on  the 
importance  of  color  with  degraded  imagery.  Without  the  benefit  of  identifying  objects  by 
profile  and  surface  features,  color  (natural  and  false)  is  significantly  important  in  the 
segmentation  of  boundaries. 

The  results  of  this  experiment  validate  previous  findings  in  that  altering  an  image 
by  modifying  the  color  disrupts  the  HVS.  Images  are  more  efficiently  processed  in 
achromatic  format  than  in  the  unnatural  state.  For  scenes  that  are  processed  globally, 
color  enhances  object  recognition. 

Whether  observing  imagery  that  is  either  degraded  or  clear,  the  role  of  natural 
color  is  significant  in  object  recognition.  When  the  color  of  the  scene  is  modified  from 
the  natural  state,  the  HVS  is  disrupted  and  therefore  is  less  efficient  in  object  recognition. 
It  is  recommended  that  scenes  be  represented  achromatically  if  natural  color  cannot  be 
attained. 
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V.  CONCLUSIONS,  AND  RECOMMENDATIONS 


The  objective  of  this  thesis  was  to  evaluate  natural  and  artificial  color  on  natural 
scene  perception.  Following  an  extensive  review  of  previous  research  concerning  the 
role  of  color  in  object  recognition,  it  was  determined  that  color  generally  played  a 
specific  role  in  enhancing  the  recognition  of  objects,  but  this  role  was  only  secondary  to 
identification  by  object  profile.  It  was  also  shown  that  depending  on  the  experimental 
methodologies  used  when  conducting  the  research,  the  role  of  color  was  often 
inconclusive.  Some  of  the  deficiencies  found  in  previous  research  experimental 
methodologies  were  the  inadequacy  of  target  availability.  Also,  "natural"  targets  were 
poorly  categorized  as  living,  man-made  or  non-living  objects,  and  the  targets  were  never 
presented  in  the  natural  environment  in  which  they  belonged.  For  instance,  an  apple  was 
placed  on  a  flat  surface  against  a  neutral  background  and  was  either  applied  color  or 
remained  as  a  line  drawing.  The  apple  was  never  located  in  a  tree  or  placed  on  the 
ground  near  the  tree  to  provide  other  visual  cues.  Tasks  were  conducted  in  a  controlled 
fashion,  normally  a  naming  or  verification  task.  Other  experiments  altered  the  edge 
definition  of  the  objects  or  provided  atypical  color  schemes  to  determine  color's  role. 
Yet,  not  one  of  these  experiment  methodologies  applied  object  recognition  in  a  global 
scene. 

The  foundation  of  this  thesis  was  to  determine  whether  the  derived  research 
conclusions  of  the  role  of  color  could  be  applied  to  objects  in  a  global  scene.  By 
selecting  images  from  an  enormous  sample  of  images  using  strict  selection  criteria,  this 
thesis  was  able  to  test  the  various  color  theories.  The  hypothesis  developed  from  the 
research  was  that  when  comparing  naturally  colored  scenes  to  false  colored  scenes. 
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participant  reaction  times  and  error  rates  would  be  different.  Using  professionally 
photographed  scenes  taken  from  various  categories,  it  was  presumed  that  a  naturally 
colored  scene  would  be  viewed  similarly  as  the  scene  in  achromatic  format.  However, 
when  the  scene  was  altered  by  changing  its  natural  state,  the  HVS  would  be  disrupted 
when  tasked  to  properly  identify  the  orientation  of  the  scene. 

A  second  hypothesis  tested  the  role  of  color  and  artificial  color  in  a  global 
environment  that  was  spatially  degraded  by  noise.  Using  the  abundance  of  scenes  that 
were  available  to  conduct  the  research,  it  was  hypothesized  that  color  would  aid  in  object 
recognition  when  compared  to  achromatic  representations  of  the  scene  but  artificially 
colored  scenes  would  hinder  the  recognition  process. 

The  results  of  both  experiments  were  conclusive.  When  comparing  prototypical 
colored  scenes  to  the  achromatic  version  of  the  scene  using  defined  edge  definitions,  it 
was  found  that  natural  color  did  not  significantly  enhance  object  recognition.  Although 
participant  reaction  times  were  slightly  smaller  for  color,  statistical  analysis  found  this 
effect  to  be  non-significant.  This  finding  confirms  the  majority  of  conclusions  found  in 
prior  research,  that  color  does  not  benefit  object  recognition.  However,  when  the  color 
was  altered  to  an  atypical  or  unnatural  representation,  participant  reaction  time  increased 
significantly;  therefore  it  was  concluded  the  effects  of  color  on  object  recognition  were 
significant.  This  important  result  is  perhaps  the  most  significant  reason  that 
contradictions  exist  from  previous  research.  Vision  scientists  often  fail  to  distinguish 
between  natural  and  artificial  color.  If  given  the  choice  of  recognizing  an  object  in  an 
atypically  colored  scene,  it  was  preferred  to  eliminate  artificial  color  from  the  scene. 
Additionally,  changing  the  level  of  brightness  to  the  scene  had  no  effect  on  participant 
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reaction  time  for  color. 


Natural  color  becomes  more  important  when  the  visual  acuity  of  the  scene 
degrades.  The  second  experiment  validated  previous  claims  on  colors  role  in  object 
recognition  for  degraded  images.  When  compared  to  the  same  scene  with  the  addition  of 
noise,  color  improved  the  ability  to  respond  to  object  recognition.  Therefore,  it  was 
concluded  that  color  enhances  object  recognition  for  degraded  scenes  when  compared  to 
a  monochromatic  representation. 

Current  military  research  in  NVD  technology  has  provided  the  technology  to 
combine  various  scenes  into  a  simple  representation  to  reduce  individual  component 
limitations  and  provide  potential  strengths.  However,  technology  has  permitted 
researchers  to  apply  color  to  image  representations  that  originate  outside  the  range  of  the 
HVS's  sensitivity.  The  ability  to  provide  standardized  colors  to  the  global  scene  is  the 
subject  of  much  skepticism  that  questions  this  feasibility.  The  conclusions  drawn  from 
this  thesis  provides  evidence  that  color  is  beneficial  when  viewing  spatially  degraded 
imagery.  However,  with  scenes  that  are  not  spatially  degraded,  natural  color  is  more 
effective  for  object  recognition  when  the  scene  is  viewed  globally. 

Further  research  is  required  to  validate  the  conclusions  of  this  study.  First,  the 
images  chosen  for  these  experiments  were  commercially  photographed  scenes.  It  is 
recommended  that  identical  procedures  be  employed  with  images  taken  from  fused 
sensors.  The  addition  of  noise  in  the  second  experiment  was  to  approximate  the  lack  of 
visual  resolution  found  in  small  military  displays.  By  utilizing  real-world  images  taken 
from  sensors  employed  in  the  field,  these  approximations  would  be  unnecessary. 

Second,  a  different  task  should  be  utilized  to  properly  identify  an  object  within  the 
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scene.  Although  this  type  of  experiment  could  potentially  limit  the  number  of  participants 
that  may  be  available,  objects  can  be  chosen  that  pertain  to  a  participant's  particular 
specialty.  Therefore,  the  participant's  level  of  expertise  could  be  tightly  controlled. 

Third,  conduct  the  viewing  on  a  display  that  approximates  smaller  military 
displays  instead  of  a  computer  color  monitor  that  contains  resolution  features  not  found 
on  field  equipment.  This  would  add  additional  realism  to  field-testing.  Finally,  by 
varying  the  degree  of  image  saturation  and  hue  manipulation,  a  determination  can  be  of 
the  amount  an  image  can  be  altered  before  participant  perceptual  degradation  occurs. 

The  concept  of  providing  color  to  fused  displays  will  one  day  enable  viewing  a 
night  time  environment  be  seen  as  a  natural  daylight  scene.  However,  much  research  is 
needed  to  validate  the  utilities  of  these  technical  achievements. 


APPENDIX  A:  COMPUTER  CODE  FOR  EXPERIMENT  ONE 

(WRITTEN  IN  ‘C’) 
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#define  .. 
# include 
tinclude 
#include 
# include 
# include 
#include 
#include 
# include 
#include 
#include 
# include 


.cdecl 
<stdio .h> 
<stdlib.h> 
<conio .h> 
<stddef .h> 
<string.h> 
<time.h> 
<vrg . h> 
<patch.h> 
<story .h> 
<keys ,h> 

<f ile .h> 


#def ine  FIXATION  1 
#def ine  STIMULI  2 

#define  NUMBER_OF_IMAGES  240 

#def ine  NUMBER_OF„PRACTICE„IMAGES  30 

//keys  ' 1'  and  ' 2 '  will  stop  interval 
//keylist  must  be  NULL  terminated 

VstoryKey  keylist[]  =  {  {  '  1 '  ,  VSTOP}, {'2r,  VSTOP}  ,  {  1  q*  ,  VSTOP} , 

{13 ,  VSTOP}, {NULL}}; 

int  Ran(int  max)  ; 

void  Permute (int*  array,  int  length); 
void  main  (void) 

{ 

short  j  ,  x ,  y ,  z ; 

char  response,  answer[3],  subject [10] , logfile [20] =  {" fin_" } ; 
long  latency; 

FILE  *fp; 

FILE  *fp2 ; 

FILE  *fp3 ; 

char  Image [NUMBER_OF_IMAGES] [10] ={ " f 001 .pcx" , " f 002 .pcx" , "f003.pcx", 
*  f004.pcx" , " f 005 .pcx" , " f006.pcx" , " f007 .pcx" , nf 008 .pcx" , ■ f009.pcx” , 

" f 010 .pcx" , "foil .pcx" , "f 012 .pcx" , " f 013 .pcx" , "f014.pcx" , " f015.pcx" , 

" f016 .pcx" , " f 017 .pcx" , "f 018 .pcx" , " f 019 .pcx" , "f020.pcx" , " f021.pcx" , 

" f 022 .pcx" , " f 023 .pcx" , " f024.pcx" , " f025 .pcx" , "f026.pcx" , " f 027 .pcx" , 

" f 028 .pcx" , " f 029 .pcx" , H f030.pcx" , "f031.pcx" , "f032 .pcx" , "f033 .pcx" , 

" f 034 .pcx" , "f035.pcx", "f036.pcx", "f037.pcx", "f038.pcx", "f039.pcx", 

" f 040 .pcx" , " f 041 .pcx" , "f 042 .pcx" , "f043 .pcx" , "f044.pcx" , "f045.pcx" , 

" f 046 .pcx" , " f 047 .pcx" , " f 048 .pcx" , " f 049 .pcx" , " f050.pcx" , " f051.pcx" , 

" f 052 .pcx" , "f053.pcx", "f054.pcx", "f055.pcx", "f056.pcx", "f057.pcx", 

" f058.pcx" , " f  0  5  9 .pcx" , "f 060 .pcx" , "f061 .pcx" , ”f062 .pcx" , " f063 .pcx" , 

" f 064 .pcx" , " f 065 .pcx" , " f 066 .pcx" , " f 067 .pcx" , " f 068 .pcx" , " f069.pcx" , 
w f 070 .pcx" , " f 071 .pcx" , " f 072 .pcx" , ■ f 073 .pcx" , " f 074 .pcx" , " f 075 .pcx" , 

" f076.pcx" , " f 077 .pcx" , " f 078 .pcx" , " f 079 .pcx" , " f080.pcx" , " f 081 .pcx" , 

" f 082 .pcx" , "f083.pcx", "f084.pcx", "f085.pcx", "f086.pcx", "f087.pcx", 
"f088.pcx" , " f 089 .pcx" , " f090.pcx" , "f091.pcx" , "f092 .pcx" , "f093 .pcx" , 

" f 094 .pcx" , " f 095 .pcx" , "f096.pcx" , "f097.pcx" , "f098.pcx" , "f099.pcx" , 

"  f 100 .pcx" , " f 101 .pcx” , " f 102 .pcx" , " fl03 .pcx" , " fl04.pcx" , " f 105 .pcx" , 
"fl06.pcx", "fl07.pcx", " f 108 .pcx" , " f 109 .pcx" , "fll0.pcx", "flll.pcx", 

"  f 112 .pcx" , * f 113 .pcx" , " fll4.pcx" , " fll5.pcx" , " fll6 .pcx" , " fll7.pcx” , 
"fll8.pcx" , " f 119 .pcx" , " f 120 .pcx" , " f 121 .pcx" , "fl22 .pcx" , "fl23 .pcx" , 

■  f 124 .pcx" , " f 125 .pcx" , " f 126 .pcx" , " f 127 .pcx" , " fl28.pcx" , " f 129 .pcx" , 
"fl30.pcx" , " fl31.pcx" , " f 132 .pcx" , "fl33 .pcx" , "fl34.pcx" , "fl35.pcx" , 
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"fl36.pcx" , "fl37.pcx”, "fl38.pcx" , "fl39.pcx" , "fl40.pcx" , "fl41.pcx" 
” fl42.pcx" , "fl43.pcx”, " fl44.pcx" , "fl45.pcx", "fl46.pcx", "fl47.pcx" 
"fl48.pcx" , "fl49.pcx" , ” f 150 .pcx" , "fl51.pcx” , "fl52 .pcx" , "fl53 .pcx" 
"fl54.pcx" , "fl55.pcx" , " fl56.pcx" , "fl57.pcx" , "fl58.pcx" , "fl59.pcx" 
"fl60.pcx" , "fl61.pcx" , " fl62.pcx" , "fl63 .pcx" , "fl64.pcx" , "fl65.pcx" 
"fl66.pcx" , " fl67 .pcx" , "fl68.pcx" , "fl69 .pcx" , "fl70.pcx" , "fl71.pcx" 
"fl72.pcx", "fl73.pcx", "fl74.pcx", "fl75.pcx", "fl76.pcx", "fl77.pcx” 
"fl78.pcx" , ” fl79 .pcx" , " f 180 .pcx" , "fl81.pcx" , "fl82.pcx" , "fl83 .pcx” 
"fl84.pcx" , " fl85.pcx" , " fl86.pcx" , "fl87.pcx", "fl88.pcx", "fl89.pcx” 
"fl90.pcx" , " fl91.pcx” , " f 192 .pcx” , "fl93.pcx" , "fl94.pcx" , "fl95.pcx" 
” fl96.pcx" , " fl97.pcx” , ■ fl98.pcx" , "fl99.pcx", "f200.pcx", "f201.pcx" 
"f202.pcx", "f203.pcx", "f204.pcx", "f205.pcx", "f206.pcx", "f207.pcx" 
"f208.pcx" , " f209.pcx" , " f210.pcx" , "f211.pcx" , "f212.pcx" , "f213 .pcx" 
"f214.pcx", "f215.pcx", " f 216 .pcx" , "f217.pcx", "f218.pcx", "f219.pcx" 
"f220.pcx" , " f221.pcx" , ” f222 .pcx" , "f223.pcx" , "f224.pcx" , ”f225 .pcx" 
"f226.pcx", "f227.pcx", "f228.pcx", "f229.pcx", "f230.pcx”, "f231.pcx" 
"f232.pcx" , " f 233 .pcx" , "f234.pcx” , "f235.pcx” , "f236.pcx" , "f237.pcx" 
"f238.pcx", "f239.pcx", "f240.pcx"}; 

char  Practice [NUMBER_OF_PRACTICE_IMAGES] [10] ={ “p4 .pcx" , "p5.pcx", 
"p6 .pcx" , "p7.pcx" , "p8.pcx" , "p9 .pcx" , "plO .pcx" , "pll.pcx" , "pl2 .pcx" 
"pl3 .pcx" , "pl4.pcx" , "pl5.pcx" , "pl6.pcx" , "pl7 .pcx" , "pl8.pcx" , 
"pl9.pcx" , "p20 .pcx" , "p21.pcx" , "p22.pcx" , "p23 .pcx" , "p24.pcx" , 
"p25.pcx" , "p26 .pcx" , "p27 .pcx" , "p28.pcx" , "p29.pcx" , "p30.pcx" , 
"p31.pcx" , "p32 .pcx" , "p33 .pcx" } ; 

char  Path [16] [50] ={ "d: \\final\\nc\\ " , "d: \\final\\ng\\" , 

"d: \\final\\fh\\" , "d: \\final\\fs\\" , 

"d: \\final\\fhs\\" , "d: \\final\\fhg\\" , 

"d:  WfinalWfsgW" ,  "d:  WfinalWfhsgW" , 

"d: \\final\\unc\\" , "d: \\final\\ung\\ " , 

"d:  \  \final\\ufh\\" ,  "d:  WfinalWufsW " , 

" d : \ \ f inal \ \ufhs \ \ "," d : \ \ final \ \uf hg\ \ " , 

"d: \\final\\ufsg\\ " , "d: \\final\\ufhsg\\ ” } ; 

char  Thislmage [100] ; 

int  c [16]  =  {0,0, 0,0,0, 0,0,0, 0,0, 0,0,0, 0,0,0); 

int  Order [NUMBER_OF_IMAGES] ,  ThisTrial; 

Vpatch_display_sequence_description  list [2] ; 

Vinit (VFULL) ; 

Vpatch_init (NULL) ; 

Vpatch_realize_message (VDISABLE) ; 

for  (y=0;  y<NUMBER_OF_IMAGES ;  y++) 

Order [y]  =  y; 

Permute ( Order ,  NUHBER_OF_IMAGES ) ; 
printf ( " \ntrying  to  open  file\n"); 

if  ( ( fp  =  Vfile_open("shw_imgs.stm" ,  VTEXT,  VREAD) )  !=  NULL) 

{ 

Vpatch_read(fp) ; 

Vfile_close (fp) ; 
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} 

else 

{ 

printf { " \nCouldn* t  open  the  .stm  file,  dude."); 
getch ( ) ; 

} 

printf (" \n\nWhat  are  this  subjects  initials?\n\ t " ) ; 
gets (subject) ; 

strcat (logfile,  subject) ; 
strcat (logfile,  Vdat"); 

fp2=f open (logfile,  "w'j  ; 

list  [0]  .interval_id  =  FIXATION; 
list [0] .keylist  =  NULL; 

list [1] . interval_id  =  STIMULI; 
list [1] .keylist  =  keylist; 

Vpa t ch_r eal i ze (FIXATION) ; 

//practice 

for (y=0 ;  y<NUMBER_OF_PRACTICE_IMAGES ;  y++) { 
printf (" \nPress  any  key  when  ready  to  initiate  trial" ); getch ( ) 
j=Ran(16) ; 

strcpy( This Image,  Path[j-1] ) ; 
strcat (This Image,  Practice [y] ) ; 

Vimage_set_.fi le  ( STIMULI ,  Thislmage)  ; 
printf ( ” \n%s" ,  Thislmage) ; 

Vinterval_set_background _pct (STIMULI ,  50.2,50.2,  50.2); 
Vpatch_realize (STIMULI) ; 

Vbeep ( ) ; 

Vpa  t  ch_di sp 1 ay_s equenc e_wa i t ( 1 i s  t ,  2 ) ; 
latency  =  list [1] .nframes ; 
response  =  list [1] .key; 
if (j<9) 

if ( (response-48) ==2) 

Verror_beep ( ) ; 

if  (j>8) 

if ( (response-48) ==1) 

Verror_beep ( ) ; 

Vpatch_unrealize (STIMULI) ; 
if  (response== ' q' ) { 
printf ( " \ntrying  to  quit" ) ; 
break ; 

} 

printf (" \npractice  trial  number  %d",  y+1) ; 

} 

y=0; 
x=0; 
z=0 ; 

for  (y=0  ;  y<NUMBER_OF_IMAGES ;  y++ )  { 

printf (" \nPress  any  key  when  ready  to  initiate  trial" ); getch ( ) 
ThisTrial  =  Order [y] ; 
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do{  j=Ran(16);}  while  ( c [ j  —  1 ]  >=  15); 
strcpy( This Image,  Path[ j-1] ) ; 
strcat (Thislmage,  Image [ThisTrial] ) ; 

Vimage_set_f ile  (STIMULI ,  Thislmage); 
c [j-l] ++; 

printf ( " \n%s" ,  Thislmage) ; 
f print f (fp2 ,  "\n%d" ,  y+1); 
f printf ( fp2 ,  ”\t%s",  Thislmage) ; 

Vinterval_set_background _pct (STIMULI ,  50.2,50.2,  50.2) 
Vpatch_realize (STIMULI)  ; 

Vbeep ( ) ; 

Vpatch_display„seguence_wait (list,  2) ; 
latency  =  list [1] .nframes; 
response  =  list [1] .key; 
if ( j<9) 

if ( (response-48) ==2 ) 

Verror_beep ( ) ; 
if  ( j>8) 

if ( (response-48) ==1) 

Verror_beep ( ) ; 

Vpatch_unrealize (STIMULI) ; 
if  (response== 1 q' ) { 
printf  ("  \ntrying  to  quit'*  ); 
break; 

printf ( " \ntrial  number  %d" ,  y+1)  ; 
fprint f ( fp2 ,  " \ t%li " ,  latency) ; 
if (j<9) 

fprintf(fp2,  " \tl"); 
if ( j>8) 

fprint f ( fp2 ,  " \t2 " ) ; 

fprint f ( fp2 ,  " \t%c " ,  response) ; 

} 

printf (" \nThe  experiment  is  over") ; 

Vpatch_unrealize (FIXATION) ; 

Vpatch_unrealize (STIMULI)  ; 

Vpatch__des troy  ( )  ; 
f close (fp2) ; 

Vexit(); 

} 

int  Ran(int  max) 

//generate  a  random  number  between  1  &  max 

{ 

double  x  =  RAND_MAX  +  1.0; 
int  y; 

y  =  1  +  rand()  *  (max/x) ; 
return  (y)  ; 

} 

void  Permute (int*  array,  int  length) 

//will  permute  a  an  array  of  length  cells 

{ 

int  a,  z,  temp; 

for  (a=0;  adength;  a++) 

{  z  =  Ran (length  -  a)  +  a; 

z — ;  //subtract  one  so  y  can  index  an  array 

temp  =  array [ z ] ; 
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APPENDIX  B:  COMPUTER  CODE  FOR  EXPERIMENT  TWO 

(WRITTEN  IN  ‘C’) 


#define  _cdecl 
#include  <stdio.h> 

#include  <stdlib.h> 

#include  <conio.h> 

# include  <stddef.h> 

#include  <string.h> 

#include  <time.h> 

# include  <vrg.h> 

#include  <patch.h> 

#include  <story.h> 

# include  <keys.h> 

#include  <file.h> 

# define  FIXATION  1 
#def ine  STIMULI  2 
# define  NUMBER_OF_IMAGES  240 
# define  NUMBER_OF_PRACTICE_IMAGES  30 

//keys  *  1*  and  ’2'  will  stop  interval 
//keylist  must  be  NULL  terminated 
VstoryKey  keylist []  =  {{’l1,  VSTOP} , 

{ '2 1 ,  VSTOP} , 

{  'q' , VSTOP} , 

{13,  VSTOP}, 

{NULL}} ; 

int  Ran(int  max) ; 

void  Permute (int*  array,  int  length); 
void  main (void) 

{ 

short  j,x,y,z,  counter=0; 

char  response,  answer[3],  subject [10] , logfile [20] ={ "jv4_" } ; 
long  latency; 

FILE  *fp; 

FILE  *fp2 ; 

FILE  *  f p3 ; 

char  Image  [  NUMBER_OF__IMAGES  ]  [10]  ={  "  f  01  .pcx" ,  "f02.pcx",  nf03  .pcx", 
"f04.pcx", B f05.pcx" , " f 06. pcx" , "f07.pcx" , " f08.pcx", " f09.pcx", 

"f 10. pcx" , " £11 .pcx" , " f 12 .pcx" , ” f 13 .pcx" , "fl4.pcx", " fl5.pcx", 

" fl6.pcx" , " fl7.pcx" , " f 18 .pcx" , "f 19 .pcx" , " f20.pcx" , " f21 .pcx" , 

" f22 .pcx" , ”f23 .pcx" , " f 24 .pcx" , " f25 .pcx" , " f26.pcx" , " f27 .pcx" , 

" f28.pcx" , " f 29 .pcx" , " f30 .pcx" , " f31.pcx" , nf32 .pcx" , "f33 .pcx" , 

" f34.pcx" , " f35.pcx" , "f36 .pcx" , "f37 .pcx" , " f38 .pcx" , "f39 .pcx" , 

" f40 .pcx" , " f41.pcx" , " f 42 .pcx" , " f 43 .pcx" , " f44.pcx" , " f45 .pcx" , 

" f46.pcx" , " f47.pcx" , " f 48 .pcx" , " f49 .pcx" , " f50.pcx" , " f51.pcx" , 

" f 52 .pcx" , " f 53 .pcx" , " f 54 .pcx" , "f55.pcx" , " f 56 .pcx" , "f 57 .pcx" , 

" f58.pcx" , " f 59 .pcx" , " f60.pcx" , " f61.pcx" , " f62.pcx" , " f 63 .pcx" , 

" f64.pcx" , " f 65 .pcx" , "f 66 .pcx" , " f 67 .pcx" , "f 68 .pcx" , " f 69 .pcx" , 

" f 70 .pcx" , " f 71 .pcx" , " f 72 .pcx" , "f73 .pcx" , "f74.pcx" , "f75.pcx”, 

" f76.pcx" , "f77.pcx" , " f78.pcx" , " f79 .pcx" , " f80.pcx" , "f 81 .pcx" , 

" f 82 .pcx" , " f 83 .pcx" , "f 84 .pcx" , " f 85 .pcx" , "f86.pcx" , " f 87 .pcx" , 
"f88.pcx" , " f 89 .pcx" , " f90.pcx" , "f91.pcx” , " f 92 .pcx" , "f93 .pcx" , 

"  f94.pcx" , " f 95 .pcx" , " f 96 .pcx" , "f97.pcx", "f98.pcx", "f99.pcx", 

"  f 100 .pcx" , " fl01.pcx" , " f 102 .pcx" , ■ f 103 .pcx" , " f 104 .pcx" , 

"  f 105 .pcx" , "fl06 .pcx" , " fl07.pcx" , " fl08 .pcx" , "fl09.pcx" , 

”  f 110 .pcx" , "fill. pcx " , "fll2.pcx" , "fll3.pcx", "fll4.pcx", 

"  f 115 .pcx" , "f 116 .pcx” , " f 117 .pcx" , " fll8 .pcx" , " f 119 .pcx" , 

"  f 120 .pcx" , " fl21.pcx" , "fl22.pcx", "fl23.pcx", "fl24.pcx". 
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"fl25.pcx" ,  "fl26.pcx" ,  ■  fl27.pcx"  ,  "  fl28.pcx" ,  "fl29.pcx", 
"fl30.pcx",  " fl31.pcx" ,  " fl32.pcx" ,  "fl33  .pcx" ,  ■  f  134 .pcx" , 

"  fl35.pcx" ,  "  fl36.pcx" ,  "fl37.pcx" ,  "fl38.pcx" , "  fl39.pcx" , 
n  fl40.pcx" , " fl41 .pcx” , ” f 142 .pcx" , " f 143 .pcx" , " f 144 .pcx" , 

" fl45.pcx" , " fl46.pcx" , " f 147 .pcx" , "fl48.pcx" , " fl49 .pcx" , 

" f 150 .pcx" , "fl51.pcx" , " fl52 .pcx" , " f 153 .pcx" , " fl54 .pcx" , 

" f 155 .pcx" , " f 156 . pcx" , * f 157 .pcx" , " f 158 . pcx" , " f 159 .pcx" , 

" fl60.pcx" , " f 161 .pcx" , " f 162 .pcx" , " f 163 .pcx" , " fl64.pcx" , 
"fl65.pcx", "fl66.pcx", "fl67.pcx", "fl68.pcx", "fl69.pcx", 
"fl70.pcx", "fl71.pcx" , "fl72.pcx" , " f 173 .pcx" , " f 174 .pcx" , 
"fl75.pcx", "fl76.pcx", "fl77.pcx", " fl78 .pcx" , " f 179 .pcx" , 

" fl80.pcx" , " fl81.pcx" , " f 182 .pcx" , " fl83 .pcx" , " f 184 .pcx" , 

" f 185 .pcx" , " f 186 .pcx" , "fl87.pcx", "fl88.pcx" , "fl89.pcx" , 

" f 190 .pcx" , " fl91 .pcx" , " f 192 .pcx" , " f 193 .pcx" , " fl94.pcx" , 

" f 195 .pcx" , " f 196 .pcx" , "fl97.pcx", "fl98.pcx" , "fl99.pcx" , 

" f 200 .pcx" , "f201.pcx" , " f 202 .pcx" , "f203.pcx" , "f204.pcx" , 

" f205 .pcx" , " f206 .pcx" , " f 207 .pcx" , "f208.pcx" , " f209 .pcx" , 

" f 210 .pcx" , " f211.pcx" , " f 212 .pcx" , "f213.pcx", "f214.pcx", 

" f 215 .pcx" , " f 216 .pcx" , " f 217 .pcx" , "f218.pcx" , ■ f219.pcx" , 

" f220.pcx" , " f221.pcx" , “ f222 .pcx" , " f223 .pcx" , " f224 .pcx" , 

" f 225 .pcx" , " f226.pcx" , " f227 .pcx" , " f228.pcx" , " f229 .pcx" , 
"f230.pcx", "f231.pcx", "f232.pcx", "f233.pcx", "f234.pcx", 
"f235.pcx", "f236.pcx", "f237.pcx", "f238.pcx", " f239.pcx", 
"f240.pcx"}; 

char  Practice [NUMBER_OF_PRACTICE_IMAGES] [10] ={ "p04 .pcx" , 

"p05 .pcx" , "p06 .pcx" , "p07 .pcx" , "p08 .pcx" , "p09.pcx" , "plO .pcx" , 
"pll .pcx" , "pl2 .pcx" , "pi 3 .pcx" # "pl4.pcx" , "pl5.pcx" , "pl6.pcx" , 
"pl7.pcx", "pl8.pcx", "pl9.pcx", "p20.pcx", "p21.pcx", "p22.pcx", 
"p23 .pcx" , "p24.pcx" , "p25 .pcx" , "p26.pcx" , "p27.pcx" , "p28 .pcx" , 
"p29 .pcx" , "p30.pcx" , "pOl .pcx" , "p02 .pcx" , "p03 .pcx" } ; 

char  Path [16] [50]=  { "d: \\f inal2\\nc\\ ” , "d: \\final2\\ng\\ " , 
"d: \\f inal2\\fc\\ " , "d: \\f inal2\\fg\\ " , 

"d: \\final2\\noise_c\\" , "d: \\f inal2\\noise_g\\ " , 

"d: \\final2\\noise_fc\\" , "d: \\f inal2\\noise_fg\\ " , 

"d: \\final2\\unc\\ " , "d: \\f inal2\\ung\\ " , 

"d: \\f inal2\\ufc\\ " , "d: \\f inal2\\ufg\\ " , 

"d: \\final2\\unoise_c\\" , "d: \\f inal2\\unoise_g\\ " , 

"d: X\f inal2\\unose_fc\\ " , "d: \\final2\\unose_fg\\ " } ; 

char  This Image [100]; 

int  c [16 ]  =  {0,0, 0,0, 0,0, 0,0,0, 0,0,0, 0,0, 0,0}; 
int  Order [NUMBER_OF_IMAGES] ,  ThisTrial; 
Vpatch__display_sequence_description  list [2] ; 

Vinit (VFULL) ; 

Vpatch_init (NULL) ; 

Vpatch_realize_message(VDISABLE) ; 

for  (y=0 ;  y<NUMBER_OF_IMAGES ;  y++) 

Order  [y]  =  y; 

Permute  ( Order ,  NUMBER_OF_IMAGES )  ; 
printf ( " \ntrying  to  open  file\n" ); 

if  ( { fp  =  Vfile_open("shw__imgs.stm" ,  VTEXT,  VREAD) )  !=  NULL) 

{ 

Vpatch_read(fp) ; 

Vfile_close (fp) ; 
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} 

else 

{ 

print f ( " \nCouldn' t  open  the  . stm  file,  dude . " ) ; getch { ) ; 

} 

printf ( " \n\nWhat  are  this  subjects  initials?\n\t " ) ;  gets (sub j ect) ; 

strcat (logfile,  subject) ; 

s treat (logfile,  " .dat" ) ; 

fp2=fopen( logfile,  "w'j  ; 

list [0] .interval_id  =  FIXATION; 

lis t [ 1 ] . interval_id  =  STIMULI ; 

list [1] .keylist  =  keylist; 

Vpatch_realize (FIXATION) ; 

printf (" \nPress  any  key  to  begin  practice  phase" ); getch () ; 
//practice 

for  (y=0 ;  y<NUMBER_OF_PRACTICE_IMAGES ;  y+  + )  { 
j=Ran(16)  ; 

strepy (This Image,  Path[j-1] ) ; 
strcat (Thislmage,  Practice [y] ) ; 

Vimage_set_file( STIMULI,  Thislmage) ; 
printf ( " \n%s " ,  Thislmage) ; 

Vinterval_jset_background__pct  ( STIMULI ,  50.2,50.2,  50.2)  ; 
Vpatch_realize (STIMULI)  ; 

Vbeep ( ) ; 

Vpatch__display_sequence_wait  (list ,  2)  ; 
latency  =  list [1] .nframes ; 
response  =  list[l].key; 
if ( j<9) 

if ( (response-48) ==2) 

Verror_beep ( ) ; 
if  (j>8) 

if ( (response-48) ==1) 

Verror_beep (); 

Vpatch_unrealize (STIMULI) ; 
if  (response==,ql ) { 
printf  ( "  \ntrying  to  cguit " )  ; 
break; 

} 

printf (" \npract ice  trial  number  %d" ,  y+1) ; 

} 

printf ( "\nPract ice  phase  over,  press  any  key  to  begin  experiment"); 

getch ( ) ; 

y=0; 

x=0; 

z=0 ; 

for  ( y=  0 ;  y<NUMBER_OF_IMAGES ;  y++ )  { 

counter++; 

if (counter%30==0)  { 

printf  ("  \nTake  a  break,  press  any  key  when  ready  to  continue"); 
getch ( ) ; 

} 

ThisTrial  =  Order [y] ; 

do{  j=Ran(16);}  while  (c[j-l]  >=  15); 
strepy (Thislmage,  Path[ j-1] ) ; 
strcat (Thislmage,  Image [ThisTrial] ) ; 

Vimage_set_file  (STIMULI,  Thislmage)  ; 
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c[  j-l]++; 

print f ( n \n%s" ,  This Image) ; 
f print f ( fp2 ,  " \n%d" ,  y+1); 
fprintf (fp2,  " \t%s" ,  Thislmage) ; 

Vinterval_set_background_pct ( STIMULI ,  50.2,50.2,  50.2) 
Vpatch_realize( STIMULI) ; 

Vbeep ( ) ; 

Vpatch_display_sequence_wait ( list ,  2 )  ; 
latency  =  list [1] .nframes; 
if ( j<9) 

if ( (response-48) ==2) 

Verror_beep ( ) ; 
if  (j>8) 

if ( (response-48) ==1) 

Verror_beep ( ) ; 

Vpatch_unrealize (STIMULI) ; 
if  (response== ’q' ) { 
printf ( ” \ntrying  to  quit"); 
break ; 

} 

printf  ("  \ntrial  number  %d",  y+1)  ; 
fprintf (fp2,  "\t%li",  latency); 

if (j<9) 

fprintf ( fp2 ,  " \tl” ) ; 

if ( j  >8 ) 

fprintf (fp2,  "\t2") ; 

fprintf (fp2,  "\t%c",  response); 

} 

printf ( "\nThe  experiment  is  over”); 

Vpatch_unrealize (FIXATION) ; 

Vpatch„ unrealize ( STIMULI ) ; 

Vpatch_destroy  ( ) ; 
f close (fp2) ; 

Vexit ( ) ; 

} 

int  Ran(int  max) 

//generate  a  random  number  between  1  &  max 

{ 

double  x  =  RAND__MAX  +  1.0; 
i n£  y; 

y  =  1  +  rand()  *  (max/x) ; 
return  (y) ; 

} 

void  Permute (int*  array,  int  length) 

//will  permute  a  an  array  of  length  cells 

{ 

int  a,  z,  temp; 

for  (a=0;  a<length;  a++) 

{z  =  Ran (length  -  a)  +  a; 

z — ; //subtract  one  so  y  can  index  an  array 
temp  =  array [z] ; 
array[z]  =  array[a] ; 
array [a]  =  temp; 

} 

} 
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APPENDIX  C:  EXPERIMENT  ONE  REACTION  TIME  AND 
ACCURACY  DATA 
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NC 

Color 

CFH 

CFS 

CFHS 

NG 

Gray 

GFH  GFS 

GFHS 

SUBJ1 

486.47 

601.43 

483.74 

533.12 

473.62 

503.69 

569.22 

513.49 

SUBJ2 

622.97 

782.38 

582.54 

712.81 

595.87 

740.81 

602.32 

663.62 

SUBJ3 

577.15 

848.55 

594.21 

660.63 

673.06 

636.65 

710.83 

790.07 

SUBJ4 

450.42 

543.12 

479.57 

483.14 

480.17 

515.90 

488.14 

505.16 

SUBJ  5 

1066.24 

1691.55 

888.34 

1620.19 

847.74 

1332.24 

1625.54 

1396.24 

SUBJ  6 

513.68 

554.78 

559.39 

629.75 

511.70 

533.12 

488.14 

559.39 

SUBJ  7 

562.00 

615.31 

558.71 

677.71 

600.32 

700.32 

615.03 

683.66 

SUBJ  8 

497.42 

536.69 

543.12 

538.89 

632.44 

542.56 

489.09 

598.57 

SUBJ  9 

613.09 

804.68 

708.61 

693.31 

742.56 

719.58 

758.59 

623.24 

SUBJ  10 

845.22 

1232.84 

1394.44 

1286.15 

993.65 

1272.27 

1091.87 

1783.22 

SUBJ  11 

463.51 

504.56- 

459.94 

486.12 

453.15 

407.53 

461.35 

468.56 

SUBJ 12 

586.99 

604.25 

631.80 

567.63 

576.99 

499.11 

565.25 

605.06 

SUBJ  13 

533.68 

641.41 

710.43 

955.17 

798.01 

701.00 

693.95 

653.63 

SUBJ 14 

504.56 

631.41 

538.25 

588.87 

519.79 

503.64 

485.92 

555.33 

SUBJ  15 

799.68 

873.46 

1135.91 

852.04 

741.37 

862.16 

864.40 

1180.30 

SUBJ  16 

712.54 

624.75 

613.45 

648.55 

614.75 

603.93 

713.41 

673.45 

SUBJ  17 

552.00 

813.96 

570.61 

598.09 

657.48 

580.54 

634.19 

635.46 

SUBJ 18 

469.46 

488.00 

543.24 

577.04 

537.61 

501.31 

515.07 

574.77 

SUBJ  19 

547.56 

678.30 

714.60 

635.86 

590.24 

638.85 

580.32 

549.19 

SUBJ  20 

616.42 

729.20 

584.38 

778.86 

1089.84 

679.86 

657.48 

897.97 

Mean  RT 

601.05 

740.03 

664.76 

726.20 

656.52 

673.75 

680.50 

745.52 

SEM 

34.05 

63.13 

51.77 

62.55 

38.13 

53.86 

60.20 

74.94 
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NC 

Color 

CFH 

CFS 

CFHS 

NG 

Gray 

GFH  GFS 

GFHS 

SUBJ1 

0.00 

0.00 

0.00 

6.67 

6.67 

0.00 

0.00 

6.67 

SUBJ2 

6.67 

13.33 

0.00 

6.67 

0.00 

0.00 

13.33 

0.00 

SUBJ3 

0.00 

0.00 

0.00 

13.33 

0.00 

0.00 

0.00 

13.33 

SUBJ4 

6.67 

0.00 

6.67 

13.33 

6.67 

0.00 

0.00 

6.67 

SUBJ5 

6.67 

0.00 

6.67 

6.67 

6.67 

0.00 

6.67 

13.33 

SUBJ  6 

0.00 

0.00 

13.33 

0.00 

6.67 

6.67 

0.00 

13.33 

SUBJ7 

0.00 

0.00 

6.67 

6.67 

0.00 

6.67 

20.00 

6.67 

SUBJ  8 

0.00 

6.67 

0.00 

6.67 

13.33 

0.00 

6.67 

6.67 

SUBJ9 

0.00 

0.00 

0.00 

6.67 

6.67 

13.33 

0.00 

26.67 

SUBJ  10 

0.00 

6.67 

0.00 

0.00 

6.67 

0.00 

13.33 

6.67 

SUBJ  11 

6.67 

6.67 

6.67 

6.67 

0.00 

13.33 

6.67 

20.00 

SUBJ  12 

0.00 

13.33 

13.33 

6.67 

0.00 

13.33 

6.67 

26.67 

SUBJ  13 

0.00 

6.67 

6.67 

20.00 

0.00 

13.33 

13.33 

0.00 

SUBJ  14 

6.67 

0.00 

13.33 

13.33 

0.00 

6.67 

0.00 

0.00 

SUBJ  15 

13.33 

6.67 

33.33 

6.67 

26.67 

20.00 

13.33 

13.33 

SUBJ  16 

13.33 

0.00 

6.67 

6.67 

0.00 

20.00 

6.67 

13.33 

SUBJ  17 

0.00 

6.67 

0.00 

0.00 

6.67 

6.67 

0.00 

6.67 

SUBJ  18 

6.67 

20.00 

6.67 

26.67 

13.33 

26.67 

20.00 

33.33 

SUBJ  19 

0.00 

6.67 

6.67 

0.00 

6.67 

13.33 

0.00 

6.67 

SUBJ  20 

6.67 

13.33 

13.33 

20.00 

20.00 

13.33 

6.67 

40.00 

Mean  RT 

3.67 

5.33 

7.00 

8.67 

i 

6.33 

8.67 

6.67 

13.00 

SEM 

1.02 

1.33 

1.78 

1.61 

1.64 

1.82 

1.53 

2.49 
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NC 

NC 

NOISE 

CFH 

CFH 

NOISE 

NG 

NG 

NOISE 

GFH 

GFH 

NOISE 

SUBJ1 

687.19 

733.30 

888.3 

1113.42 

730.53 

806.52 

978.81 

783.3 

SUBJ2 

1122.73 

1402.17 

1283.84 

1522.72 

1211.62 

1040.24 

1154.95 

1574.34 

SUBJ3 

521.41 

723.58 

671.4 

639.72 

613.44 

626.36 

683.95 

862.84 

SUBJ4 

510.54 

613.17 

523.79 

673.58 

505.93 

791.64 

776.08 

631.04 

SUBJ5 

720.48 

720.8 

745.11 

769.14 

751.25 

697.7 

797.88 

669.97 

SUBJ6 

677.75 

754.46 

750.57 

1250.78 

799.97 

833.3 

979.82 

742.28 

SUBJ7 

802.19 

808.3 

934.58 

1221.11 

919.41 

880.27 

933.3 

893.55 

SUBJ8 

584.59 

621.09 

652.2 

722.75 

679.83 

621.77 

769.41 

676.9 

SUBJ9 

1028.53 

1220.51 

1145.79 

1521.96 

919.19 

1300.78 

1147.18 

1513.83 

SUBJ  10 

698.31 

660.09 

993.55 

848.68 

781.52 

821.12 

801.25 

874.32 

SUBJ 11 

815.44 

864.85 

728.82 

883.99 

849.37 

745.8 

987.78 

940.24 

SUBJ  12 

596.64 

598.19 

706.38 

751.89 

745.48 

754.83 

751.08 

608.9 

SUBJ  13 

807.19 

799.41 

855.32 

927.94 

888.65 

1046.93 

847.88 

1186.62 

SUBJ  14 

687.19 

733.3 

888.3 

1113.42 

730.53 

806.52 

978.81 

783.3 

SUBJ  15 

879.54 

1013.43 

856.98 

1087.55 

932.34 

1056.87 

901.23 

1013.54 

MEAN  RT 

742.65 

817.78 

841.66 

1003.24 

803.94 

855.38 

899.29 

917.00 

SEM 

44.72 

59.15 

50.67 

74.33 

42.34 

47.56 

36.38 

76.77 
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NC 

NC 

NOISE 

CFH 

CFH 

NOISE 

NG 

NG 

NOISE 

GFH 

GFH 

NOISE 

SUBJ1 

0 

0 

13.33 

0 

0 

6.67 

13.33 

6.67 

SUBJ2 

0 

0 

0 

0 

0 

20 

0 

6.67 

SUBJ3 

6.67 

20 

6.67 

13.33 

13.33 

20 

13.33 

26.67 

SUBJ4 

0 

20 

6.67 

20 

6.67 

13.33 

0 

26.67 

SUBJ5 

13.33 

33.33 

13.33 

33.33 

13.33 

26.67 

13.33 

26.67 

SUBJ  6 

0 

13.33 

6.67 

33.33 

13.33 

26.67 

20 

13.33 

SUBJ7 

0 

20 

13.33 

13.33 

0 

26.67 

20 

13.33 

SUBJ  8 

0 

13.33 

0 

0 

20 

13,33 

0 

13.33 

SUBJ  9 

6.67 

0 

6.67 

20 

13.33 

33.33 

20 

20 

SUBJ  10 

0 

6.67 

13.33 

13.33 

6.67 

13.33 

13.33 

13.33 

SUBJ  11 

6.67 

6.67 

13.33 

20 

6.67 

33.33 

13.33 

20 

SUBJ  12 

0 

6.67 

13.33 

13.33 

13.33 

20 

0 

6.67 

SUBJ  13 

0 

0 

6.67 

6.67 

6.67 

26.67 

20 

0 

SUBJ  14 

0 

0 

13.33 

0 

0 

6.67 

13.33 

6.67 

SUBJ  15 

0 

6.67 

6.67 

13.33 

0 

13.33 

13.33 

13.33 

MEAN  Err 

2.22 

9.78 

8.89 

13.33 

7.56 

20.00 

11.55 

14.22 

SEM 

1.06 

2.59 

1.25 

2.84 

1.70 

2.25 

2.00 

2.14 
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